There's an interesting follow on debate in the comments of Reza's last post "The Trouble with Scientific Data".
Enjoy!
Tuesday, March 11, 2008
Saturday, March 8, 2008
The Trouble with Scientific Data
There are two things I'd like to discuss, but we'll split up the blog posts for brevity. The first is the trouble with scientific data, and the second is the problem with toxicity testing as it stands today (I'll give you a hint, it has to do with dose). Let's start with scientific data.
After reading Paul's most recent post I realized it would be prudent to discuss data (no not the cyborg on Star Trek). More specifically, discuss scientific data. Why? We need to come to an agreement here on the degree of accuracy for facts contained in this presentation. After a brief overview of how we get from raw data to the statements in this presentation I think you'll agree that a debate on these facts, in this format, is the equivalent of lowering global sea level using a 5 gallon bucket.
The source page data, and the links and footnotes contained herein are largely from NGO (non-governmental organization) websites. Others are simply summaries/interpretations of data on other websites, all from a 30,000 ft view of the science. The problem here is that the "data" on these source pages really are not data at all. Merely black and white summaries of someone's interpretation of data... which was presented in a summarized and interpreted form for them, which was summarized and interpreted for that person, who read it in the primary literature... which is merely a summary and interpretation of data. I knew we would get to the data eventually. Just like a game of telephone, the message can be altered drastically from its initial form. More importantly, the initial message (the primary literature) may have been a complete crock because of the poor quality of the science.
So let's go down the lifecycle of data starting at the end (the presentation in question):
1) There are over 100,000 synthetic chemicals in commerce today. Only a handful of these have even been tested for human health impacts and NONE of them have been tested for synergistic health impacts, that means when they interact with all the other chemicals we’re exposed to every day.22 - Annie Leonard: Story of Stuff
I know for a fact (but only because this is my field) that this statement is sheer distortion and parts of it are utterly false. But, let's look up her footnote #22 (that is the 22 at the end of that sentence). 22 leads us to...
2)“Of the more than 80,000 chemicals in commerce, only a small percentage of them have ever been screened for even one potential health effect, such as cancer, reproductive toxicity, developmental toxicity, or impacts on the immune system. Among the approximately 15,000 tested, few have been studied enough to correctly estimate potential risks from exposure. Even when testing is done, each chemical is tested individually rather than in the combinations that one is exposed to in the real world. In reality, no one is ever exposed to a single chemical, but to a chemical soup, the ingredients of which may interact to cause unpredictable health effects.” From Coming Clean Campaign’s Body Burden
information, retrieved 11/8/07 from http://www.chemicalbodyburden.
org/
Ok, so Annie got her information from the Coming Clean Campain's "Body Burden" information on chemicalbodyburden.org. I'd like to point something out here. Annie isn't getting her data from scientists at places like the National Academy of Sciences, or the National Toxicology Program, or even primary literature. Her information is coming from a group who's sole existence depends on lobbying for a particular cause. It doesn't seem like exactly an unbiased place to get one's information. Not exactly balanced information. Roughly the equivalent of watching Fox News to get a balanced viewpoint. But let's go to the site and see where they got the data to make that statement.
3) http://chemicalbodyburden.org/whatisbb.htm
Under the heading "What is Body Burden", about half way down you will find the statement that was used for Annie's proof. The reference for this statement? Non-existent. In fact, looking over this website gave me chills in general. There is a high-degree of spin and stretching of facts, applying facts incorrectly, and falsehoods. So how can we backtrack the data now? Well, if you go to the about us link on the page of this organization (http://chemicalbodyburden.org/aboutus.htm) you'll see that their scientific toxicology staff consists of 1 person, namely: Dr. Harmon, PhD, Toxics Campaign Scientist
Dr. Harmon is a member of greenpeace, and is helping out with this organization. I am sure that Dr. Harmon is a excellent scientist, but even the best scientist in the world would have trouble keeping up to date with all of the changing literature and all of the data needed to support the generalized statements made on the website. There is simply too much out there to be an expert of all.
So it looks like the reference trail has gone dry. But let's imagine where the data possible came from. We can put a big black box.
4) The Black Box
The black box here is the X number of individuals/interpretations between the statements on the chemical body burden site (#3) and primary literature (#5). So step 4 is potentially step 4, 4.1, 4.2, 4.21, 4.22 etc... before we get to step 5. Who knows how facts get changed here.
5) Finding Primary Literature
AKA Publications. If we do a NCBI pubmed search (http://www.ncbi.nlm.nih.gov/sites/entrez) on BFR, for example, we get 498 hits. Here is one example of the 498 hits: http://www.ncbi.nlm.nih.gov/pubmed/18315715?ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
Enjoy reading the abstract. After that, read all of the pages of that publication. Now read the other 498 references. I'll wait.
After that read about dioxins. That'll produce 13,275 hits.
Actually, that is really too specific, do a search of toxic and you'll get 142,197 hits.
Why don't we start with reading 1 paper
6) Reading Primary Literature
For each of these publications you'll have to read the author's interpretation of the data, and look at the raw data yourself to determine if you agree with the author's assessment. In addition, the raw data in publications isn't really raw, but presented in a certain way by the author.
Based on my calculations, we have between 6-492 people standing between Annie's initial statement (remember Annie, she is in the story of stuff) and the data. That is a disturbing game of telephone.
Bottom line. You can find scientists who fall on either side of every single argument and fact in science. On top of this, many of the things Annie is saying is simply incorrect, as is her source. If you'd like to read something on mixtures, I recommend this: http://www.ncbi.nlm.nih.gov/pubmed/16081521?ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
Therefore, having a debate on the validity of these facts is not a useful exercise. Suffice to say that there has been tremendous research on mixtures over the years, and it seems to be overall a phantom issue since only those with common MOA should even be in the realm of concern. On top of all of this is the subject of the next posting : The Dose.
After reading Paul's most recent post I realized it would be prudent to discuss data (no not the cyborg on Star Trek). More specifically, discuss scientific data. Why? We need to come to an agreement here on the degree of accuracy for facts contained in this presentation. After a brief overview of how we get from raw data to the statements in this presentation I think you'll agree that a debate on these facts, in this format, is the equivalent of lowering global sea level using a 5 gallon bucket.
The source page data, and the links and footnotes contained herein are largely from NGO (non-governmental organization) websites. Others are simply summaries/interpretations of data on other websites, all from a 30,000 ft view of the science. The problem here is that the "data" on these source pages really are not data at all. Merely black and white summaries of someone's interpretation of data... which was presented in a summarized and interpreted form for them, which was summarized and interpreted for that person, who read it in the primary literature... which is merely a summary and interpretation of data. I knew we would get to the data eventually. Just like a game of telephone, the message can be altered drastically from its initial form. More importantly, the initial message (the primary literature) may have been a complete crock because of the poor quality of the science.
So let's go down the lifecycle of data starting at the end (the presentation in question):
1) There are over 100,000 synthetic chemicals in commerce today. Only a handful of these have even been tested for human health impacts and NONE of them have been tested for synergistic health impacts, that means when they interact with all the other chemicals we’re exposed to every day.22 - Annie Leonard: Story of Stuff
I know for a fact (but only because this is my field) that this statement is sheer distortion and parts of it are utterly false. But, let's look up her footnote #22 (that is the 22 at the end of that sentence). 22 leads us to...
2)“Of the more than 80,000 chemicals in commerce, only a small percentage of them have ever been screened for even one potential health effect, such as cancer, reproductive toxicity, developmental toxicity, or impacts on the immune system. Among the approximately 15,000 tested, few have been studied enough to correctly estimate potential risks from exposure. Even when testing is done, each chemical is tested individually rather than in the combinations that one is exposed to in the real world. In reality, no one is ever exposed to a single chemical, but to a chemical soup, the ingredients of which may interact to cause unpredictable health effects.” From Coming Clean Campaign’s Body Burden
information, retrieved 11/8/07 from http://www.chemicalbodyburden.
org/
Ok, so Annie got her information from the Coming Clean Campain's "Body Burden" information on chemicalbodyburden.org. I'd like to point something out here. Annie isn't getting her data from scientists at places like the National Academy of Sciences, or the National Toxicology Program, or even primary literature. Her information is coming from a group who's sole existence depends on lobbying for a particular cause. It doesn't seem like exactly an unbiased place to get one's information. Not exactly balanced information. Roughly the equivalent of watching Fox News to get a balanced viewpoint. But let's go to the site and see where they got the data to make that statement.
3) http://chemicalbodyburden.org/whatisbb.htm
Under the heading "What is Body Burden", about half way down you will find the statement that was used for Annie's proof. The reference for this statement? Non-existent. In fact, looking over this website gave me chills in general. There is a high-degree of spin and stretching of facts, applying facts incorrectly, and falsehoods. So how can we backtrack the data now? Well, if you go to the about us link on the page of this organization (http://chemicalbodyburden.org/aboutus.htm) you'll see that their scientific toxicology staff consists of 1 person, namely: Dr. Harmon, PhD, Toxics Campaign Scientist
Dr. Harmon is a member of greenpeace, and is helping out with this organization. I am sure that Dr. Harmon is a excellent scientist, but even the best scientist in the world would have trouble keeping up to date with all of the changing literature and all of the data needed to support the generalized statements made on the website. There is simply too much out there to be an expert of all.
So it looks like the reference trail has gone dry. But let's imagine where the data possible came from. We can put a big black box.
4) The Black Box
The black box here is the X number of individuals/interpretations between the statements on the chemical body burden site (#3) and primary literature (#5). So step 4 is potentially step 4, 4.1, 4.2, 4.21, 4.22 etc... before we get to step 5. Who knows how facts get changed here.
5) Finding Primary Literature
AKA Publications. If we do a NCBI pubmed search (http://www.ncbi.nlm.nih.gov/sites/entrez) on BFR, for example, we get 498 hits. Here is one example of the 498 hits: http://www.ncbi.nlm.nih.gov/pubmed/18315715?ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
Enjoy reading the abstract. After that, read all of the pages of that publication. Now read the other 498 references. I'll wait.
After that read about dioxins. That'll produce 13,275 hits.
Actually, that is really too specific, do a search of toxic and you'll get 142,197 hits.
Why don't we start with reading 1 paper
6) Reading Primary Literature
For each of these publications you'll have to read the author's interpretation of the data, and look at the raw data yourself to determine if you agree with the author's assessment. In addition, the raw data in publications isn't really raw, but presented in a certain way by the author.
Based on my calculations, we have between 6-492 people standing between Annie's initial statement (remember Annie, she is in the story of stuff) and the data. That is a disturbing game of telephone.
Bottom line. You can find scientists who fall on either side of every single argument and fact in science. On top of this, many of the things Annie is saying is simply incorrect, as is her source. If you'd like to read something on mixtures, I recommend this: http://www.ncbi.nlm.nih.gov/pubmed/16081521?ordinalpos=2&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
Therefore, having a debate on the validity of these facts is not a useful exercise. Suffice to say that there has been tremendous research on mixtures over the years, and it seems to be overall a phantom issue since only those with common MOA should even be in the realm of concern. On top of all of this is the subject of the next posting : The Dose.
Thursday, March 6, 2008
Annotated Script with Sources
The Story of Stuff website provides references and sources for their statements in the form of an annotated script for the video. The script can be found on their references page.
1) On the "100,000 chemicals and synergistic impacts" statement: I tracked the source of that statement (using the source noted in the annotated script to this page.) This appears to be a web site focused on pointing out what we don't know about the chemicals in circulation in the US. It's not as well sourced as I'd like especially on the specific statement about the number of synergistic impact studies that have been conducted. The lack of a secondary source doesn't necessarily make it wrong, but I'm definitely interested in seeing the studies you mentioned that suggest we do understand the synergistic impacts.
2) The sources on BFRs are on page 5 of the script. The sites they point to point to other studies. I read a bunch of them and my take away is that BRFs are leaching into the environment (wildlife) and people's bodies. The amounts are small (in people) but have increased about an order of magnitude in the last 30 years. Plenty of alternatives exist.
The thing I struggle with is the idea that flame retardants are going to save my life. If I'm still asleep in my bed while my bedroom is on fire I'm going to be much more concerned with smoke inhalation than getting burned by my bed.
3) A few of the BFR sources describe the breast milk studies. Unless the sources are just wrong the statement she's making is correct. They make a point to state that Breast milk is still the best choice and they even site sources to that point. (See page 5.)
4) Factory Workers: The point made in the video is that this is a global system. One of the challenges for US factory workers is they cost more and have more restrictions as a result of the safety standards required in the US. I think the safety standards are a great idea, but they are not a global standard occurrence. So while I agree that this does not appear to be a major issue in the States, countries like China count and if they are anything like the US during our industrial revolution then factory workers are getting the short end of the stick.
I'm out of time for this post so I'll follow up on your other points in a few days.
1) On the "100,000 chemicals and synergistic impacts" statement: I tracked the source of that statement (using the source noted in the annotated script to this page.) This appears to be a web site focused on pointing out what we don't know about the chemicals in circulation in the US. It's not as well sourced as I'd like especially on the specific statement about the number of synergistic impact studies that have been conducted. The lack of a secondary source doesn't necessarily make it wrong, but I'm definitely interested in seeing the studies you mentioned that suggest we do understand the synergistic impacts.
2) The sources on BFRs are on page 5 of the script. The sites they point to point to other studies. I read a bunch of them and my take away is that BRFs are leaching into the environment (wildlife) and people's bodies. The amounts are small (in people) but have increased about an order of magnitude in the last 30 years. Plenty of alternatives exist.
The thing I struggle with is the idea that flame retardants are going to save my life. If I'm still asleep in my bed while my bedroom is on fire I'm going to be much more concerned with smoke inhalation than getting burned by my bed.
3) A few of the BFR sources describe the breast milk studies. Unless the sources are just wrong the statement she's making is correct. They make a point to state that Breast milk is still the best choice and they even site sources to that point. (See page 5.)
4) Factory Workers: The point made in the video is that this is a global system. One of the challenges for US factory workers is they cost more and have more restrictions as a result of the safety standards required in the US. I think the safety standards are a great idea, but they are not a global standard occurrence. So while I agree that this does not appear to be a major issue in the States, countries like China count and if they are anything like the US during our industrial revolution then factory workers are getting the short end of the stick.
I'm out of time for this post so I'll follow up on your other points in a few days.
Subscribe to:
Posts (Atom)