17 October 2008

RCTs “placed on an undeserved pedestal” – head of NICE

I can’t find any other reference to this information apart from as reported by Pharmatimes:

[update: also here and The Independent  ‘Statistics can help but doctors must also use their judgement’ which includes the pleasing statement: ‘It is scientific judgement – conditioned by the totality [my bold] of the evidence – that lies at the heart of making decisions about the benefits and harms of therapeutic interventions]

‘The chairman of the UK’s National Institute for Health and Clinical Excellence (NICE) has suggested randomised controlled trials (RCTs) should no longer be seen as the be-all and end-all of clinical research.

In a speech last night to the Royal College of Physicians, Professor Sir Michael Rawlins said such studies had been placed “on an undeserved pedestal”. He called for other types of research, including observational studies, to be given greater attention.

Professor Rawlins presides over an organisation that has regularly indicated its discontent with clinical evidence supplied by drug manufacturers. For its part, industry has been vocal in its criticisms of NICE’s cost-effectiveness models. More recently, Professor Rawlins has sharply criticised industry pricing practices
for new drugs.

All the same, some may be surprised at his willingness to question the value of RCTs, generally seen as the most rigorous tests for a new medicine, and talk up the benefits of other types of study.

In his speech Professor Rawlins said clinical trials were:

* Virtually impossible to conduct properly when studying treatments for rare diseases with very few patients

* Often prohibitively expensive. He cited a recent study of 153 trials completed in 2005 and 2006, which showed a median cost of over £3 million, with one trial costing £95 million. One manufacturer has estimated that the average cost per patient of a clinical trial rose from £6,300 in 2005 to £9,900 in 2007

* Even “unnecessary” when, as in the case of Novartis’ Glivec (imatinib) for chronic myeloid leukaemia , a treatment produced a particularly “dramatic” benefit

However, Professor Rawlins also expressed concern about the growing tendency, especially in cancer research, for clinical trials to be stopped early.

“The desire to stop trials early is understandable, but the possibility that an interim analysis is a ‘random high’ may be difficult to avoid,” he said. Moreover, there was “no consensus among statisticians as to how best to handle the problem”.

Prof Rawlins also had some criticism for his medical colleagues, many of whom adopted too rigid an approach to clinical research, he claimed, particularly in the trend towards ranking different types of clinical trial in terms of importance.

Hierarchies attempt to replace judgment with an over-simplistic, pseudo-quantitative, assessment of the quality of the available evidence,” he commented.

Accoording to Professor Rawlins, observational studies, historical controlled trials and case-control studies are also important sources of information.

What is needed is for “investigators to continue to develop and improve their methodologies; for decision-makers to avoid adopting entrenched positions about the nature of evidence; and for both to accept that the interpretation of evidence requires judgment“, he concluded.’



  1. The full text is available from http://www.rcplondon.ac.uk/pubs/contents/304df931-2ddc-4a54-894e-e0cdb03e84a5.pdf

    Comment by Howard White — 17 October 2008 @ 5:54 pm

  2. Quote from full report: “As Bradford Hill, the architect of the RCT, stated
    so cogently: ‘Any belief that the controlled trial is the only way would mean not that
    the pendulum had swung too far but that it had come right off the hook’.”

    Comment by homeopathy4health — 17 October 2008 @ 6:39 pm

  3. And “As William Blake (1757–1827) observed: ‘God forbid that truth should be confined to
    mathematical demonstration’.”

    Comment by homeopathy4health — 17 October 2008 @ 6:41 pm

  4. “Experiment, observation and mathematics – individually and collectively – have a
    crucial role to play in providing the evidential basis for modern therapeutics.
    Arguments about the relative importance of each are an unnecessary distraction.”

    Comment by homeopathy4health — 17 October 2008 @ 6:42 pm

  5. “For those with lingering doubts about the nature of evidence itself, I remind them
    that while Gregor Mendel (1822–84) developed the monogenic theory of inheritance
    on the basis of experimentation,215 Charles Darwin (1809–82) conceived the theory of
    evolution as a result of close observation,216 and Albert Einstein’s (1879–1955) special
    theory of relativity217 was a mathematical description of aspects of the world around
    us. William Harvey’s discovery of the circulation of the blood – as he described in
    De Motu Cordis – was based on an elegant synthesis of all three forms of evidence.”

    Comment by homeopathy4health — 17 October 2008 @ 6:43 pm

  6. Professor Sir Michael Rawlins is giving examples where RCT’s are impossible (to few patients) or prohibitively expensive (large clinical trials). These are PRAGMATIC problems, where we might have to settle for less perfect data to enable something to be done for the sufferers of rare disease or to curb the already high cost of drug development.

    In your excerpt he also criticises the practice of stopping a trial early, to which we might add the practice of extending them, in order to catch a randomly good week in which to terminate the study.

    None of this is relevant to homeopaths objections to RCT’s which are weird, hand waving, pseudo-philosophical objections that, at their worst, make spurious appeal to quantum theory.

    You are claiming apparently miraculous cures for a range of complaints from everyday infections and bruises through unpleasant acute infections to serious chronic conditions. You would think you could cheaply and efficiently organise a DBRCT with such a range of complaints to choose from. There is also Andies blinded proving challenge which seems a reasonable test of at least a facet of your claims.

    What I keep saying is that if there were any merit to homeopathy I would expect there to be a body of work from the last half century or so using a variety of experimental designs to probe homeopaths efficacy. There would for example be observational studies to establish some idea which would led on into RCT’s some of which would be positive and some negative. The research articles would relate to each other and complement each others weaknesses with their own strengths, taken together they would build a convincing case for homeopathy.

    There is in fact a body of shockingly awful literature in the Alt Med journals containing poorly presented data from flawed experiments that relate to each other very little. If you give this data it’s due and pull out as much info as you can, you find the totality of evidence suggests homeopathy is no better than placebo. Nothing that Professor Rawlins has said will help you avoid this conclusion.

    I find it interesting that having posted something from one scientist you should go on to quote several other authorities in your own comments section. Is this another case of a homeopath confusing authority with reasoned argument? Do you think that taking a selection of utterances from great men and unattributed sources then putting them together amounts to evidence for the efficacy of homeopathy?

    Comment by Derrik — 21 October 2008 @ 12:48 pm

  7. Tbe ‘other authorities’ are quoted by ‘the scientist’ in the full report, I suggest you read all of it like I did, the link is there.

    ‘There is in fact a body of shockingly awful literature in the Alt Med journals containing poorly presented data from flawed experiments that relate to each other very little. If you give this data it’s due and pull out as much info as you can, you find the totality of evidence suggests homeopathy is no better than placebo’. Have you actually done that yourself Derrik?

    You are showing poor logic Derrik, ‘if there was any merit to homeopathy, there would be da-de-da-de-da..’. I can’t help it if science is WAAAAY behind, sorry.

    Comment by homeopathy4health — 21 October 2008 @ 1:06 pm

  8. h4h, if, as you state, science is behind in its understandings then why don’t you tell science how homeopathy works so it can investigate it? Or has it already done that, we’ve had ultra-dilutions – scientifically untenable, memory of water – scientifically untenable, quantum entanglement – scientifically untenable. What’s next?

    Comment by gimpy — 21 October 2008 @ 1:52 pm

  9. ‘scientifically untenable’…because?

    Comment by homeopathy4health — 21 October 2008 @ 2:41 pm

  10. The section just before your Blake quote reads:

    “Such judgements relate to the extent to which each of the components of the evidence base is ‘fit for purpose’. Is it reliable? Does it appear to be general sable? Do the intervention’s benefits outweigh its harms?”

    So we exersise our judgment about the bundle of anecdotes homeopaths are so fond of:

    Are a bundle of anecdotes fit for purpose in assessing the efficacy of homeopathy? – no. Are anecdotes reliable? – no. Are anecdotes generalisable? – no. Do the intervention’s benefits outweigh its harms? – hmm possibly as there are no direct harms but their are also no benefits above placebo.

    I think you found a fragment of the on going self reflective debate about the interpretation of clinical evidence, one of the characteristics of evidence based medicine, noticed that it raised a couple of the issues with RCT’s and decided it therefore helped your case. You were wrong.

    Critically when Professor Rawlins says you need to use judgment he doesn’t mean you are free to pluck any interpretation of the data out of the air and claim your judgment is as good as any other.

    Since you ask I’ve read various meta-analysis and read several articles from the journal homeopathy. One of the weird things about that journal is the proportion of actual research to opinion pieces, another is the reporting of statistical test results in tables rather than more informative graphical presentation of data. Can’t you guys draw graphs?

    At the dawn of modern science homeopathy was apparently as good an idea as any other. It fell out of favour not because of some wired sectarian split but because its predictions did not accord well with reality. If it had accorded well with reality it would have been studied and the literature would have grown as a result along with a research community studying the phenomina. Homeopathy is a living fossil from the pre-scientific era not an advanced form of medicine ahead of its time.

    However for the game of the thing, why not tell us which of the drawbacks the prof draws our attention to make RCT’s inappropriate for homeopathy.

    Comment by Derrik — 21 October 2008 @ 3:12 pm

  11. It hasn’t fallen out of favour Derrik, you need to check your reality. It’s growing in popularity around the world.

    Comment by homeopathy4health — 21 October 2008 @ 3:27 pm

  12. Anyone who tries to study and research it gets the Randi treatment…

    Comment by homeopathy4health — 21 October 2008 @ 3:28 pm

  13. By fallen out of favour I obviously mean with medical researchers and academic scientists rather than the general population.

    I guess your trying to say that people don’t study homeopathy because there is no funding available and they are afraid of being ridiculed by their colleagues.

    That makes no sense historically. Homeopathy was part of the early medical establishment in the UK, which is why retains a presence in the NHS, albeit with reduced funding as its ineffectiveness becomes clear to NHS managers. From that position you would think research would have been easily managed. It doesn’t make sense in modern times either. Good research doesn’t require letters after your name and a grant from a research body all it requires is diligent measurement and analysis and clear reporting of data. It wouldn’t have cost those reporting their experiments in Homeopathy any more money to do their experiments properly; perhaps it was simply beyond their ability to do so.

    If you could reproducibly perform Andies “tell 30c remedies apart by the symptoms they cause in you, a healthy informed homeopath” trick, you would be well on your way to demonstrating there is something worth looking at in homeopath. You could try that un-blinded at work. If you can do it unblended you could get a receptionist to blind it for you. If you can still manage this feet then people want to hear from you. You don’t need a research grant to do this.

    Comment by Derrik — 21 October 2008 @ 4:04 pm

  14. I don’t believe that would do the trick either. Randi would be sent for….I am now thoroughly skeptical of skeptics.

    Have a look at the sites Closedminded Science and Pathological Disbelief (see blogroll).

    Comment by homeopathy4health — 21 October 2008 @ 4:10 pm

  15. Tell you what why don’t you go and buy some homeopathic pills (any, but take 30Cs) and take one every 10 minutes for a whole day and report back? [updated: the same remedy not different ones]

    Comment by homeopathy4health — 21 October 2008 @ 4:14 pm

  16. And that reminds me, what DID happen to skeptic (Sam) Nash who went off to the Royal London Homeopathic Hospital to investigate homeopathy…..I see his blog has disappeared….(skepticbunker)

    Comment by homeopathy4health — 21 October 2008 @ 4:23 pm

  17. “Tell you what why don’t you go and buy some homeopathic pills (any, but take 30Cs) and take one every 10 minutes for a whole day and report back?”

    I think taking a lactose pill every 10 mins for a day would make me feel sick, half a mars bar usually achieves that much. Is there anyting more spesific you might predict?

    “I don’t believe that would do the trick either. Randi would be sent for….I am now thoroughly skeptical of skeptics.”

    The thing is it isn’t about doing the trick; it is about being curious about the world. If I were you I would be doing it anyway, just to see if it could indeed be done. I concede that if you were able to do it then to convince sceptics you would have to demonstrate it in front of adjudicators we both trust but that is hardly an insuperable obstacle. With regards to public relations, if you could pull this trick repeatedly so that no reasonable person could doubt the truth of the claim and still sceptics refused to believe you would be able to paint us as reactionary nay-sayers with ease. You could humiliate us in the court of public opinion from just one little demo! Why not try?

    I’ve had a look at Brian D. Josephson’s lecture and his website. His continued existence at one of the UK’s premier university surely mitigates against the idea the scientists are an entirely closed minded bunch. There is a difference however between being open minded and being happy for a form of medicine that clinical evidence suggests to be ineffective and which is biochemically implausible to be sold to the general public.

    Comment by Derrik — 21 October 2008 @ 5:13 pm

  18. you can get non-lactose pills. Why not try?

    ‘You could humiliate us in the court of public opinion from just one little demo’. I doubt it.

    ‘that clinical evidence suggests to be ineffective’ but is found to be effective in the real world such that several countries actively use it.

    Comment by homeopathy4health — 21 October 2008 @ 5:28 pm

  19. h4h, I don’t think you will find any countries that preferentially use homeopathy, nor will you find your arguments that the miracle of homeopathy is being suppressed by the magician Randi taken seriously until you can demonstrate that it works.

    Comment by gimpy — 21 October 2008 @ 9:14 pm

  20. Derrik, does plain water make you sick?

    YOu can take one lactose pill, put it into a pet bottle with mineral water, say, 330 cc one, from which a slight amount of water is poured out to allow for mixing the water well before each sip, and take a small sip (my patients take one tea-spoonful as a dose) from this bottle every 10 minutes shaking it vigorously before each sip – just as homeopathy4health has suggested.

    That would contain only 1 pill – I’m not sure how much a sugar pellet from Helios weighs, but it cannot be much, the rest is just water, also taking some 50 teaspoonfuls 1 teasponnful at a time cannot be really much and won’t ruin your kidneys – if this will be the next objection to this simple experiment.

    Do this for a day, and tell us the next day what happened. You may tell us without telling us the name of the remedy – make it so that we guess what the remedy was? That can be fun!


    Comment by ez — 22 October 2008 @ 6:46 am

  21. Addition – I am not joking, if you are sensitive enough to the remedy that you’ll take you’ll be likely to prove it and show its most characteristic symptoms, so this might be a good way to try to distinguish two unnamed pills – something that M Simpson always asks for. However, you – and we – shoudl keep in mind that you’ll be sensitive to some remedies and not so much to others, in which case we will not find spotting the remedy that you’ll be proving that easy.

    Comment by ez — 22 October 2008 @ 6:52 am

  22. Ez! Nice to speak with you again.

    OK that’s an interesting protocol and I can’t see how the delivery method would have an effect on me, I might just try it. However you have left yourself some get outs. If I’m not sensitive enough, or I’m not sensitive to that remedy, then I won’t be able to pick out the symptoms.

    This is why I think you guys should try this, because you have the skills and the knowledge to ensure it is carried out with some chance of a positive results, if that is possible at all. You know what symptoms to expect from these remidies right? You believe you can detect these symptoms right? You already know which remedies you think you are sensitive right? You also know which remedies will provoke symptoms different enough from each other to be easily distinguished.

    What I find so strange is that this demonstration sounds so trivial I would expect something like it to be an early test of competence at homeopathy school. When this is proposed I always think it’s so strange that you don’t say something like:

    “Sure we did that at the end of first year and second year. Do you know what, I misidentified Arnica in my second year! How embarrassing! But I correctly identified the other five remedies out of the 300 we were supposed to know so I got a good mark! Here is a list of remidies I could confidently tell apart…”

    This neve happens, any idea why?

    Comment by Derrik — 22 October 2008 @ 8:48 am

  23. Because to do so deranges health, it’s not pleasant or fun.

    Comment by homeopathy4health — 22 October 2008 @ 10:04 am

  24. Well, Derrik,

    thank you for a nice greeting, and what we get at our school is cases of real person solved by a real homeopath and the person has responded curatively to the remedy given to them, but we are given only the tape case or recording of the interview with some additions about how the person looked and behaved during the interview. We have to analyse the case, repertorise etc., choose a remedy and explain why we chose that remedy – so far I have correctly “guessed” (with all the necessary analysis) about 10 remedies, only missed once, but in that case what I chose was a complementary remedy, so I cannot say that I was a complete failure. The answers are carefully checked by a tutor – a homeopath with experience, – and if a person consistently fails to make proper analysis which usually also means that they get ther remedies wrong, they do not receive marks for the study units and have to re-do them…

    Do you really want to hear the list of remedies that I got right? Stramonium, Phosphoric Acid, Argentum Nitricum, Lachesis, Lilium Tigrinum, China, a couple of others which I did several years ago so I cannot recall them right away, and I mistook Medorrhinum for Thuja, well if this helps.

    Yes, I think homeopathy4health’s comment is relevant and an important addition to the protocol that we suggested above is that you have to stop taking your doses as soon as you confidently feel that you have some sort of a strong reaction, otherwise you might get all sort of aggravations and might need your remedy antidoted by a good homeopath – through taking your case, which you would not like to do, I guess, so please be sure to adhere to this one point if you ever decide to try it out.

    Re “the get-away”, well, the solution is that you have to do the same protocol with several remedies, say 3-4 remedies from something like a Helios Basic First-aid kit, which contains the remedies that are used rather often, so many people are likely to respond to them, and the chance that you’ll hit a remedy to which you are sensitive enough will be obviously higher.

    To permit us to judge about the remedy you need to faithfully record everything unusual or strongly experienced symptom/sign/sensation – chilliness/ sensation of heat, strangely thirsty etc. Well, what’d you say?

    Comment by ez — 22 October 2008 @ 11:03 am

  25. I think we could manage this. Suppose you were to give me a list of 6 remedies, from which I would ask the pharmacist to pick 2 out for me and blind them as I described above. This is not perfect but I don’t want to spend more time or money on this than that. Then you only have 6 to guess from, which makes life a little easier for you, and you can write the list so that the likely symptoms are different from each other and the remidies likely to have an effect on a healthy but insensitive adult. I won’t look them up so I won’t know what I am expected to experience from each and either hide or imagine the symptoms when I record them. I will also notify you that I’m doing some rather boring typing at the moment so things inducing finger and shoulder aches, eye strain etc. may be difficult to identify against the background noise.

    In your example from homeopathy school you identified which remedy should be given to a subject on the basis of their symptoms, rather than identified a remedy given to a healthy person given a particular remedy. I understand that makes sense if your trying to teach people to hand out a particular remedy in response to a particular list of symptoms but that doesn’t test whether the handed out remedy has any effect. We are going to use a little consequence of the theory to test its validity by seeing if remedies do indeed induce symptoms in a healthy person.

    Oh what potency should I use?

    Comment by Derrik — 22 October 2008 @ 11:58 am

  26. Good point about the background noise. Provers are asked for their ‘usual’ symptoms before a proving starts, so you might need to screen anything usual for you out of your report.

    Comment by homeopathy4health — 22 October 2008 @ 1:35 pm

  27. As homeopathy4health has suggested, and I also think it appropriate, 30 C potency would probably the best place to start.

    “healthy but insensitive adult” – well, that’s why not one but many people usually take part in a proving, and only those that clearly display symptoms “above the noise” are chosen for the next stage of the provings, when they are given still higher potencies so that they could produce more symptoms.

    I think we should ask homeopathy4health for a list of some 6 remedies, as she is the owner of the blog, if you are really going to proceed with this?

    And the “background” symptom report will have to be prepared over a period longer than one day, maybe a week or at least 2-3 days would be good?

    Comment by ez — 23 October 2008 @ 12:11 am

  28. My selection:

    Apis, Belladonna, Carbo veg, Rhus Tox, Mag Phos, Ipecac

    What do you think of those ez, would you substitute any?

    Comment by homeopathy4health — 23 October 2008 @ 8:21 am

  29. Homeopathy4health,

    I think it is an interesting selection of clearly different remedies that cover quite different ranges of susceptibility, so they should be good for our tentative proving test, as they should be relatively easy to distinguish without much in-depth (high potency) provings.

    So – to Derrik – you will ask the pharmacist to pick out 2 remedies from this list without telling you which is which, and – how are you going to proceed? It would be good to be sufficiently meticulous with protocols – after all it’s rigor in documentation that is required to make the results easy to interpret?

    First try one, for one day or until symptoms appear, and then if you’ll turn out to be insensitive to the first, try the other? Please, confirm this! It would be wise to wait for at least a week if you end up taking several doses but no unusual symptoms appear on that day… Water doses are usually milder in action and may take time to develop – like LM’s in a way.

    It would also be good if we get enough of the modalities if something crops up – that is, if, for example, a pain appears, would it be felt less on warming it up or on cooling it down – or none of this, changes in thirst and food cravings – well, what one would usually need for an acute prescription, – but it is essential that ALL unusual and strong experiences, – including strange ideas and moods, – be recorded.

    Any more comments, homeopathy4health?

    Comment by ez — 23 October 2008 @ 1:29 pm

  30. For each unusual experience I suggest that Derrik writes down:

    The time, where it is in/on his body and if it spreads anywhere else where to, how it feels in as much detail as possible (use metaphors – ‘as if being…’), how he is feeling in himself moodwise/emotionally, what makes it feel better and what makes it feel worse. What it looks like if he can see it. Plus as you say any difference in thirst and food preferences and any difference in sleep pattern. Any of these.

    Comment by homeopathy4health — 23 October 2008 @ 2:39 pm

  31. Oh, and also if any of his ‘background noise’ gets better…

    Comment by homeopathy4health — 23 October 2008 @ 2:41 pm

  32. Right, I totally agree. Hopefully we have not missed anything important.

    (Thank you very much for the discussion!)

    So we’ll have to wait for Derrik’s response now.

    Comment by ez — 23 October 2008 @ 2:46 pm

  33. I will do this I think. I agree that the protocol needs to be clear first so I will write something detailed first and post it somewhere to give you a chance to comment and make changes in advance.

    I think a quick word about the maths is important. I did quite carefully select the two out of six combinations for the following reason.

    If you have to pick two things out of six then there are 35 ways of doing so. There are 6 options for the first selection and 5 options for the second selection and 6×5=35!

    The rest will be clearer if I write these out explicitly, let the 6 remedies be denoted by the letters A-F. The possible combinations are:


    Which is a nicely intuitivly delt with 5 by 6 matrix.

    So you will pick the correct combination, if for example it were AB, by chance alone 1 time in 35, which is a little better than the 1 time in 20 for the conventional p-value cut off of 0.05, but not nearly so exacting a task as the more stringent 1 time in a 100 cut off of 0.01. That seemed quite a fair p-value to demand.

    However, there is a problem with my choice of numbers, you will get one or other but not both correct of other remedies correct 8 times in 35, that is the last 4 members of the top row of my matrix plus the 4 combinations CB, DB, EB and FB. This 8 in 35 is closer to 1 time in 4 than to 1 time in 5. Toss a coin twice and you will get 2 head 1 time in 4. That is quite common and not even slightly statistically significant. This means that the experiment has what we might call low power ie it isn’t very sensitive. Perhaps a small mistake on your part, or insensitivity on my part, will return a false negative result. This is a legitimate criticism of my experimental design but one you should decide to accept or challenge before we try it! Perhaps you can think of a more powerful alternative without over burdening my time or costing me to much in pills?

    My other concern is that we need to decide who makes the final call on the remedy. Suppose you and Ez and another homeopath all make different selections, you could cover most of the options available and almost guarantee that one of you is right. How will you deal with this?

    I appreciate that you getting this wrong isn’t going to change anything; I’m just having fun with ideas. It’s just its more fun to play with details.

    Comment by Derrik — 23 October 2008 @ 5:25 pm

  34. Oh

    Could you replace the Carbo Veg. I looked it up once to see why you thought it was a cure for scurvy. I know I won’t know what I’m taking but its probably best if I know nothing about any of the alternatives, and I am totally ignorant about the others.

    Apart from Rhus Tox is used for chicken pos.. but I havn’t read the simillium thing about it.

    Comment by Derrik — 23 October 2008 @ 5:28 pm

  35. So just to clarify before I consider the intricacies of your probability analysis, you want to do two provings?

    If so, I am surprised because by doing one and noticing changes shows that a 30C further diluted in even more water taken repeatedly has an effect when skeptics would say such a thing is implausible. Don’t we need just to do the one to start with?

    Comment by homeopathy4health — 23 October 2008 @ 6:16 pm

  36. I suggest replacing the Carbo Veg with Lycopodium and the Rhus Tox with Bryonia:

    Apis, Belladonna, Lycopodium, Bryonia, Mag Phos, Ipecac.

    Comment by homeopathy4health — 23 October 2008 @ 6:51 pm

  37. Homeopathy4health,

    Yes, it is better if the remedies are totally unknown, so let’s replace them as you suggest.

    And I have to echo the question to Derrik – what is the exact purpose of your experiment – not to criticise you but to help discuss the protocol.

    You are going to see if you get any “health” alterations upon taking a “sub-molecular” soultion of something – in which case just one proving would be enough,

    or to accept this as already known/established/supposedly true/as an axiom of a sort and see whether such alterations are reproducibly effected by the remedies in different people, taking you as an example, for which you would ask a homeopath (yes, I also think that 3 homeopaths would be better, but you would also be checking their qualifications as homeopaths more than anything with this design, well, I don’t mind a test, but I would not like to have you experience “health” problems just to see if we are good enough to tell the remedies apart?) to figure out (it is not just “guessing”, we’ll have to repertorise and study the totality of your symptoms, just as if we were taking a patient’s case, unless you produce some clear key-notes which are well known) what those remedies are? Then it is better to do two provings and see if the new symptoms that you experience are not just “random reaction”, which you would attribute to “placebo” effect of just taking something which is supposed to do something, no doubt, but constitute a coherent pattern that a homeopath sufficiently trained would be able to recognise.

    (Sorry it’s a bit lengthy, I hope you see what I mean?)

    Comment by ez — 23 October 2008 @ 11:43 pm

  38. I’m sorry; I’ve made a ridiculous mistake!

    6×5=30 !!!!!!!!!

    So a 1 in 30 chance of identifying both remedies correctly by chance alone, an 8 in 30 chance of identifying one of the remedies correctly.


    Comment by Derrik — 24 October 2008 @ 9:17 am

  39. So does the experiment still make statistical sense to you?

    Comment by homeopathy4health — 26 October 2008 @ 10:58 am

  40. Hello again. Sorry I have been very busy.

    I do think the experiment makes statistical sense, it remains however only one experiment. It would hardly be the most conclusive thing in the world. However one of the things about experiments is that after you do them, you look at the data, have another think, and design a better one.

    I also think we can use this to think about what Professor Sir Michael Rawlins was talking about. The experiment we have sketched out above is of the “frequentist” school of thought. See the way I work out how likely an event is to occur by chance alone, then think, if I would expect it to occur less frequently than 1 time in 20 by chance alone, but it then goes on to occur anyway, that there is cause and effect at work rather than chance. Each such experiment returns a single piece of information. Usually several experiments must be combined in a single study to enable a meaningful interpretation of what is going on in the world. There are ways of combining such information from several such studies, meta-analysis for example, but they essentially stand alone. DBRCT’s are usually this kind of experiment.

    Sir Michael Rawlins would like us to consider Baysian approaches as well. Where the frequentist view of probability concerns the frequency of particular events in well defined situations, Baysian statistics considers probability as a measure of ones state of knowledge. These are quite different ways of thinking about statistics. Consider the question: What is the probability that dogs live on a planet orbiting the star Sirius? How could a frequentist approach such a question? Some would argue that for totally unknown situations the probability of 0.5 should be assigned. That doesn’t make sense here, we know dogs are earth bound, we think they evolved on earth, its unlikely a world inhabitable by any mammal orbits Sirius etc etc. Here a Baysian approach makes more sense and can bundle up all our previous knowledge and give odds near inf:1 against that dogs live in the Sirus star system.

    The problem with this approach is deciding what counts as prior knowledge and deciding what probabilities to assign to it. Frequentist probabilities are absolute, degrees for assent to propositions are fuge-able. Rawlins describes a Baysian treatment of the “GREAT” study comparing mortality in patients requiring anti-blood clotting being given them at home or after transfer to hospital. This treatment interprets the result of the study against the backdrop of what they call “prior knowledge”, which amounts to the statisticians estimate of what amounts to a plausible outcome. On the one hand this provides a way of tempering the implausibly positive results of the study with prior knowledge, on the other in this case it privileges a statisticians hunch over hard data. Imagine if a small increase in mortality was detect in the study and treated in the same way, the statisticians hunch would still have won out. What subtle manipulations could big pharma inflict on such studies if they were to be widely adopted?

    Having said that Baysian statistics do closely mirror how we deal with information in real life. In practice I don’t think in terms of frequencies of events occurring under tightly defined conditions, I give degrees of assent to propositions. Perhaps the designers of Rawlins GREAT study should have done a Baysian analysis first and realised their study would provide insufficient extra information to have an impact on clinitions opinions formed on the basis of already available information and either abandoned the study or made it big enough to have an impact

    Anyway this is where we get back to our little experiment. Obviously if the result is positive to you, I will think a 1 in 30 random event is more likely than all the rest of my knowledge about biochemistry being wrong so I won’t change my mind. Equally if the result is negative for you wouldn’t consider that to out way all the case studies you have already accorded so much trust to. Actually I think your response would be quite sensible, the experiment is underpowered and the “false negative rate” is probably quite high.

    Having said all that, I would love to have a go. I think it would be fun and a welcome distraction from my otherwise tedious thesis writing duties. Unfortunately, in the cold light of day, I don’t think I have time to play with in this way, just at the moment. However, if I have a couple of weeks together when I could keep a health diary, one week to get a background and the next two weeks, one for each remedy, to record any effects, then I will definitely give it a go and let you know.

    I did start thinking about a health diary actually, it’s amazing how many aches and pains you suddenly notice if you focus on yourself. I suspect literal and metaphorical naval gasing is the greatest H&S hazard of this experiment.

    Do you know anywhere on the internet where I could look at typical homeopathic symptom recording notes so I can try to give you my results in a form as simple as possible for you to interpret?

    Comment by Derrik — 27 October 2008 @ 6:50 pm

  41. Derrik,

    Well, I think, if you just follow the suggestions about the notes to take that homeopathy4health has mentioned in the post No.30 (and No.31), then it would be just fine!

    Please, let us know when you start!

    Still, it would be interesting to know how you plan to deal with the problem that might arise when you have got the symptoms of the remedy right, but we (well, I) failed to recognise it because of poor knowledge/lack of clinical practice? There must be a way to account for that… Maybe, it makes sense to ask more homeopaths to take part in the assessment, as you say, at least, more than two. I could ask only one person (homeopath) I know well to help out here, what would you think? (Maybe, homeopathy4health has any suggestions as well?)

    Also we probably have to decide how to organise the exchanges related to this experiment before you actually start?

    But after all you may be right that we should just start somewhere, and then work on improving it, rather than making it too cumbersome from the beginning.

    Comment by ez — 28 October 2008 @ 1:20 pm

  42. No harm in trying visit us Bodypeace Complementary & Alternative Medicine
    Offers a wide range of treatments provided by highly qualified practitioners who are members of their relevant professional associations and societies.

    Comment by Marvin Blake — 7 November 2008 @ 5:52 am

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: