DSM-5 controversy: Lane on “SAD”, Frances Follows the Money, Spitzer et al on Kappa reliability

DSM-5 controversy round up:

Lane on “SAD”; Frances “Follows the Money”; Spitzer et al on Kappa reliability; A Closer Look at Pending Changes to the Future of Psychiatric Diagnosis June issue The Journal of Nervous and Mental Disease

Post #178 Shortlink: http://wp.me/pKrrB-2em

Side Effects at Psychology Today

From quirky to serious, trends in psychology and psychiatry.

Christopher Lane, Ph.D. | June 11, 2012

Naming an Ailment: The Case of Social Anxiety Disorder”

“Social phobia” vs. “Social anxiety disorder”: What’s in a name?

…“Using data collected from a telephone survey of residents of New York State,” the letter writers continue, “we investigated whether the disorder name affects the perceived need for treatment. Random-digit dialing was used to obtain phone numbers … In total, 806 people participated.”

“Respondents heard a brief vignette describing a person who experiences discomfort in social situations and often avoids social events. These symptoms were labeled as either social phobia or social anxiety disorder, and respondents indicated whether the person should seek mental health treatment.”

The results are dubious to say the least…

Read full commentary

Psychology Today

DSM 5 in Distress | Allen Frances

Follow The Money
APA puts publishing profits above public trust

Allen Frances MD | June 11, 2012

…APA treats DSM-5 like a valuable publishing property, not as a public trust that importantly impacts on people’s lives and public policy. It is excellent at protecting its “intellectual property” with confidentiality agreements and at protecting its trademark and copyright with bullying threats of law suits. But APA has been sadly incompetent and wildly profligate in the day-to-day work of actually producing a safe and scientifically sound DSM-5.

Dr Scully is asking us to believe ten very unbelievable things. My view – if you want to understand why an unreliable and unsafe DSM-5 is being rushed prematurely to market – is to “follow the money…”

Read full commentary at DSM-5 in Distress



The American Journal of Psychiatry, VOL. 169, No. 5

Letters to the Editor | May 01, 2012

Standards for DSM-5 Reliability

Robert L. Spitzer, M.D.; Janet B.W. Williams, Ph.D.; Jean Endicott, Ph.D.
Princeton, N.J. New York City

Am J Psychiatry 2012;169:537-537. 10.1176/appi.ajp.2012.12010083

TO THE EDITOR: In the January issue of the Journal, Helena Chmura Kraemer, Ph.D., and colleagues (1) ask, in anticipation of the results of the DSM-5 field trial reliability study, how much reliability is reasonable to expect. They argue that standards for interpreting kappa reliability, which have been widely accepted by psychiatric researchers, are unrealistically high…

A Closer Look at Pending Changes to the Future of Psychiatric Diagnosis

Released: 6/7/2012 9:00 AM EDT
Source: Wolters Kluwer Health: Lippincott Williams & Wilkins

Articles Have Potential to Affect Final DSM-5 Standards as Public comment Period Ends

Newswise — New York, NY (June 7, 2012) – The June issue of The Journal of Nervous and Mental Disease (JNMD) features a special section focused on the impending release of the revised Diagnostic and Statistical Manual of Mental Disorders (DSM-5), an update to psychiatric diagnosis standards. JNMD is published by Lippincott Williams & Wilkins, part of Wolters Kluwer Health.

The Journal of Nervous and Mental Disease Editor-in-Chief John A. Talbott, MD, (a past APA president and DSM-III collaborator) comments in his editorial, “The timing of this special section on DSM-5, therefore, is particularly auspicious because it provides the potential for these articles to affect the final DSM-5 decisions.” The DSM-5 manual, currently scheduled for publication in May 2013, is going through its final public comment period through June 15, 2012.

Many articles within the section present criticisms of DSM-5 proposals. Specifically, several authors worry that the new DSM-5 standards may open up more opportunities for false-positives – a doctor diagnosing a condition when it is not present, or providing medication when it is not needed.

• “Diagnostic Inflation: Causes and Suggested Cure” by Batstra and Frances displays the authors’ concern that the proposed changes to DSM-5 will result in diagnostic inflation and inappropriate use of medication. They suggest “stepped diagnosis,” which includes a watch-and-wait period before beginning medication, to combat false-positives.

• In “Recurrence of Bereavement-Related Depression: Evidence for the Validity of the DSM-IV Bereavement Exclusion From the Epidemiological Catchment Area Study,” Wakefield and Schmitz contend that the DSM-5 proposal to remove the bereavement exclusion from the definition of a major depressive episode would cause those who are experiencing normal grief after the death of a loved one to be mislabeled as clinically depressed.

Other articles respond to DSM-5 proposals to include new disorders and diagnostic constructs. For example, DSM-5 proposes to reclassify pathological gambling as a behavioral addiction, which may pave the way for other excessive behaviors to be included in this construct in the future.

• Mihordin takes a look at the potential consequences of this change in his article, “Behavioral Addiction V Quo Vadis?” in which he presents hypothetical criteria for the diagnosis of pathological model railroading disorder.

• Good and Burstein respond to the DSM-5 proposal to include a hebephilic subtype to the diagnosis of pedophilia in “Hebephilia and the Construction of a Fictitious Diagnosis”. Additionally, Wakefield examines two DSM-5 proposals on classifying pathological forms of grief as mental disorders in “Should Prolonged Grief Be Reclassified as a Mental Disorder in DSM-5? Reconsidering the Empirical and Conceptual Arguments for Complicated Grief Disorder.”

Included in the special section, “Psychotropic Marketing Practices and Problems: Implication for DSM-5” by Raven and Perry looks at how certain aspects of DSM-5 could be used by the pharmaceutical industry as marketing tools, especially with a wider customer base resulting from false-positive patients. In “A Critique of the DSM-5 Field Trials,” Jones examines problems that may have compromised the usefulness of the DSM-5 field trials.

It is important to note that the articles in the special section of JNMD were written at various points since February 2010 based on the criteria sets posted on the DSM-5 website. Many of these criteria sets have been updated since their initial posting. “Thus, the critiques of certain proposals contained in these articles may no longer be fully relevant to what is actually being proposed for DSM-5,” Dr. Talbott states in his editorial. Visit the DSM-5 website at http://www.dsm5.org/  for the most accurate information on what is being considered for inclusion in DSM-5.

# # #

About The Journal of Nervous and Mental Disease
Founded in 1874, The Journal of Nervous and Mental Disease is the world’s oldest, continuously published independent scientific monthly in the field of human behavior. Articles cover theory, etiology, therapy, social impact of illness, and research methods


Somatic Symptom Disorder could capture millions more under mental health diagnosis

Somatic Symptom Disorder could capture millions more under mental health diagnosis

Post #172 Shortlink: http://wp.me/pKrrB-29B

By Suzy Chapman | Dx Revision Watch

Update: My submission to the Somatic Symptom Disorder Work Group in response to the third DSM-5 draft and stakeholder review can be read here: Chapman Response to Third Draft DSM-5 SSD Proposals

May 26, 2012

While media and professional attention has been focused on the implications for introducing new disorders into the DSM and lowering diagnostic thresholds for existing categories, the Somatic Symptom Disorders (SSD) Work Group has been quietly redefining DSM’s Somatoform Disorders with radical proposals that could bring millions more patients under a mental health diagnosis.

The SSD Work Group is proposing to rename the Somatoform Disorders section of DSM-IV to “Somatic Symptom Disorders,” eliminate four existing  DSM-IV categories: somatization disorder [300.81], hypochondriasis [300.7], pain disorder*, and undifferentiated somatoform disorder [300.82] and replace them with a single new category – “Somatic Symptom Disorder.”

*In DSM-IV: Pain Disorder associated with a general medical condition (only): Psychological factors, if present, are judged to play no more than a minimal role. This is not considered a mental disorder so it is coded on Axis III with general medical conditions.See http://behavenet.com/pain-disorder for definitions and criteria for other DSM-IV presentations of Pain disorder.  For DSM-5, it appears that all presentations of Pain disorder will be subsumed under the new SSD category.

If approved, these proposals will license the application of a mental health diagnosis for all illnesses – whether “established general medical conditions or disorders” like diabetes, heart disease and cancer or conditions presenting with “somatic symptoms of unclear etiology” – if the clinician considers the patient is devoting too much time to their symptoms and that their life has become “subsumed” by health concerns and preoccupations, or their response to distressing somatic symptoms is “excessive” or “disproportionate,” or their coping strategies “maladaptive.”

Somatoform Disorders – disliked and dysfunctional

The SSD Work Group, under Chair, Joel E. Dimsdale, MD, says current terminology for the Somatoform Disorders is confusing and flawed; that no-one likes these disorders and they are rarely used in clinical psychiatric practice. Primary Care physicians don’t understand the terms and patients find them demeaning and offensive [1,2].

The group says the terms foster mind/body dualism; that the concept of “medically unexplained” is unreliable, especially in the presence of medical illness, and cites high prevalence of presentation with “medically unexplained somatic symptoms” (MUS) in general medical settings – 20% in Primary Care, 40% in Specialist Care, 33-61% in Neurology; that basing a diagnosis of psychiatric disorder on MUS alone is too sensitive.

The Work Group might have considered dispensing altogether with a clutch of disliked, dysfunctional categories. Instead, the group proposes to rebrand these disorders and assign new criteria that will capture patients with diverse illnesses, expanding application of psychiatric services, antidepressants and behavioural therapies like CBT, for the “modification of dysfunctional and maladaptive beliefs about symptoms and disease, and behavioral techniques to alter illness and sick role behaviors.”

Focus shifts from “medically unexplained” to “excessive thoughts, behaviors and feelings”

The Work Group’s proposal is to deemphasize “medically unexplained” as the central defining feature of this disorder group.

For DSM-5, focus shifts to the patient’s cognitions – “excessive thoughts, behaviors and feelings” about the seriousness of distressing and persistent somatic (bodily) symptoms – which may or may not accompany diagnosed general medical conditions – and the extent to which “illness preoccupation” is perceived to “dominate” or “subsume” the patient’s life.

“[The SSD Work Group’s] framework will allow a diagnosis of somatic symptom disorder in addition to a general medical condition, whether the latter is a well-recognized organic disease or a functional somatic syndrome such as irritable bowel syndrome or chronic fatigue syndrome…” [3]

“…These disorders typically present first in non-psychiatric settings and somatic symptom disorders can accompany diverse general medical as well as psychiatric diagnoses. Having somatic symptoms of unclear etiology is not in itself sufficient to make this diagnosis. Some patients, for instance with irritable bowel syndrome or fibromyalgia would not necessarily qualify for a somatic symptom disorder diagnosis. Conversely, having somatic symptoms of an established disorder (e.g. diabetes) does not exclude these diagnoses if the criteria are otherwise met…” [4]

To meet requirements for Somatization Disorder (300.81) in DSM-IV, a considerably more rigorous criteria set needed to be fulfilled: a history of many medically unexplained symptoms before the age of thirty, resulting in treatment sought or psychosocial impairment. The diagnostic threshold was set high – a total of eight or more medically unexplained symptoms from four, specified symptom groups, with at least four pain and two gastrointestinal symptoms.

In DSM-5, the requirement for eight symptoms is dropped to just one.

One distressing symptom for at least six months duration and one “B type” cognition is all that is required to tick the box for a bolt-on diagnosis of a mental health disorder – cancer + SSD; angina + SSD; diabetes + SSD; IBS + SSD…

The most recent proposals for new category “J 00 Somatic Symptom Disorder.”

Note that the requirement for “at least two from the B type criteria” for the second draft has been reduced to “at least one from the B type criteria” for the third iteration of draft proposals. This lowering of the threshold is presumably in order to accommodate the merging of the previously proposed “Simple Somatic Symptom Disorder” category into the “Complex Somatic Symptom Disorder” category, a conflation now proposed to be renamed to “Somatic Symptom Disorder.” No revised “Disorder Description” and “Rationale/Validity” documents reflecting the changes made between draft two and draft three were issued for the third and final draft.

Ed: Update: Following closure of the third stakeholder review on June 15, 2012, proposals, criteria and rationales were frozen and the DSM-5 Development website was not updated to reflect any subsequent revisions. Proposals, criteria and rationales, as posted for the third draft in May 2012, were removed from the DSM-5 Development website on November 15, 2012 and placed behind a non public log in. Consequently, criteria as they had stood for “Somatic Symptom Disorder” at the point at which the third draft was issued can no longer be accessed but are set out on Slide 9 in this presentation, which note, does not include three, optional Severity Specifiers that were included with the third draft criteria. Since any changes to the drafts are embargoed in preparation for publication of DSM-5, in May 2013, I cannot confirm whether any changes have been made to the draft subsequent to June 15, 2012.

IASP and the Classification of Pain in ICD-11  Prof. Dr. Winfried Rief, University of Marburg, Germany

Slide 9

Rief Presentation ICD-11 Pain

How are highly subjective and difficult to measure constructs like “Disproportionate and persistent thoughts about the seriousness of one’s symptoms” and “Excessive time and energy devoted to these symptoms or health concerns” to be operationalized?

By what means would a practitioner determine how much of a patient’s day spent “searching the internet looking for data” (to quote an example of the SSD Work Group Chair) might be considered a reasonable response to chronic health concerns and what should be coded as “excessive preoccupation” or indicate that this patient’s life has become “subsumed” or “overwhelmed” by concerns about illness and symptoms? One hour day? Two hours? Three?

At the APA’s Annual Conference earlier this month, SSD Work Group Chair, Joel E. Dimsdale, presented an update on his group’s deliberations. During the Q & A session, an academic professional in the field expressed concern that practitioners who are not psychiatric professionals or clinicians might have some difficulty interpreting the wording of the B type criteria to differentiate between negative and positive coping strategies.

Dr Dimsdale was asked to expand on how the B type criteria would be defined and by what means patients with chronic medical conditions who devote time and energy to health care strategies to try to improve their symptoms and level of functioning would be evaluated in the field by the very wide range of DSM users; how would these patients be differentiated from patients considered to be spending “excessive time and energy devoted to symptoms or health concerns” or perceived as having become “absorbed” by their illness?

I am not persuaded by Dr Dimsdale’s reassurances that his Work Group will try to make this “crystal clear” in the five to six pages of manual text in the process of being drafted for this disorder chapter. Nor am I reassured that these B (1), (2) and (3) criteria can be safely applied outside the optimal conditions of field trials, in settings where practitioners may not necessarily have the time for, nor instruction in administration of diagnostic assessment tools, and where decisions to code or not to code may hang on arbitrary and subjective perceptions of DSM end-users lacking clinical training in the use of the manual text and application of criteria.

Implications for a diagnosis of SSD for all patient populations

Incautious, inept application of criteria resulting in a “bolt-on” psychiatric diagnosis of a “Somatic Symptom Disorder” could have far-reaching implications for all patient populations:

Application of highly subjective and difficult to measure criteria could potentially result in misdiagnosis with a mental health disorder, misapplication of an additional diagnosis of a mental health disorder or missed diagnoses through dismissal and failure to investigate new or worsening somatic symptoms.

Patients with cancer and life threatening diseases may be reluctant to report new symptoms that might be early indicators of local recurrence, metastasis or secondary disease, for fear of attracting a diagnosis of “SSD” or of being labelled as “catastrophisers.”

Application of an additional diagnosis of Somatic Symptom Disorder may have implications for the types of medical investigations, tests and treatments that clinicians are prepared to consider and which insurers prepared to fund.

Application of an additional diagnosis of Somatic Symptom Disorder may impact payment of employment, medical and disability insurance and the length of time for which insurers are prepared to pay out. It may negatively influence the perceptions of agencies involved with the assessment and provision of social care, disability adaptations, education and workplace accommodations.

Patients prescribed psychotropic drugs for perceived unreasonable levels of “illness worry” or “excessive preoccupation with symptoms” may be placed at risk of iatrogenic disease or subjected to inappropriate behavioural therapies.

For multi-system diseases like Multiple Sclerosis, Behçet’s syndrome or Systemic lupus it can take several years before a diagnosis is arrived at. In the meantime, patients with chronic, multiple somatic symptoms who are still waiting for a diagnosis would be vulnerable.

The burden of the DSM-5 changes will fall particularly heavily upon women who are more likely to be casually dismissed when presenting with physical symptoms and more likely to receive inappropriate antidepressants and anti-anxiety medications for them.

Proposals allow for the application of a diagnosis of Somatic Symptom Disorder where a parent is considered excessively concerned with a child’s symptoms [3]. Families caring for children with any chronic illness may be placed at increased risk of wrongful accusation of “over-involvement” with a child’s symptomatology.

Where a parent is perceived as encouraging maintenance of “sick role behaviour” in a child, this may provoke social services investigation or court intervention for removal of a sick child out of the home environment and into foster care or for enforced in-patient “rehabilitation.” This is already happening in families with a child or young person with chronic illness, notably with Chronic fatigue syndrome or ME. It may happen more frequently with a diagnosis of a chronic childhood illness + SSD.

Dustbin diagnosis?

Although the Work Group is not proposing to classify Chronic fatigue syndrome, IBS and fibromyalgia, per se, within the Somatic Symptom Disorders, patients with CFS – “almost a poster child for medically unexplained symptoms as a diagnosis,” according to Dr Dimsdale’s presentation – or with fibromyalgia, irritable bowel syndrome, chronic Lyme disease, Gulf War illness, chemical injury and chemical sensitivity may be particularly vulnerable to misapplication or misdiagnosis with a mental health disorder under these SSD criteria.

There is considerable concern that this new Somatic Symptom Disorder category will provide a “dustbin diagnosis” in which to shovel the so-called “functional somatic syndromes.”

15% of “diagnosed illness” and 26% of “functional somatic” captured by SSD criteria

For testing reliability of CSSD criteria, three groups were studied for the field trials:

488 healthy patients; a “diagnosed illness” group of 205 patients with cancer and malignancy (some in this group were said to have severe coronary disease) and a “functional somatic” group comprising 94 people with irritable bowel and “chronic widespread pain” (a term used synonymously with fibromyalgia).

Patients in the study were required to meet one to three cognitions: Do you often worry about the possibility that you have a serious illness? Do you have the feeling that people are not taking your illness seriously enough? Is it hard for you to forget about yourself and think about all sorts of other things?

Dr Dimsdale reports that if the response was “Yes – a lot.” then [CSSD] was coded.

15% of the cancer and malignancy group met SSD criteria when “one of the B type criteria” was required; if the threshold was increased to “two B type criteria” about 10% met criteria for dual-diagnosis of diagnosed illness + Somatic Symptom Disorder.

For the 94 irritable bowel and “chronic widespread pain” study group, about 26% were coded when one cognition was required; 13% coded with two cognitions required.

Has the SSD Work Group produced projections for prevalence estimates and potential increase in mental health diagnoses across the entire disease landscape?

Did the Work Group seek opinion on the medico-legal implications of missed diagnoses?

Has the group factored for the clinical and economic burden of providing CBT for modifying perceived “dysfunctional and maladaptive beliefs about symptoms and disease, and behavioral techniques to alter illness and sick role behaviors” in patients for whom an additional diagnosis of Somatic Symptom Disorder has been coded?

Where’s the science?

Dr Dimsdale admits his committee has struggled from the outset with these B type criteria but feels its proposals are “a step in the right direction.”

The group reports that preliminary analysis of field trial results shows “good reliability between clinicians and good agreement between clinician rated and patient rated severity.” In the trials, CSSD achieved Kappa values of .60 (.41-.78 Confidence Interval).

Kappa reliability reflects agreement in rating by two different clinicians corrected for chance agreement – it does not mean that what they have agreed upon are valid constructs.

Radical change to the status quo needs grounding in scientifically validated constructs and a body of rigorous studies not on pet theories and papers (in some cases unpublished papers) generated by Dr Dimsdale’s work group colleagues.

Where is the substantial body of independent research evidence to support the group’s proposals?

“...To receive a diagnosis of complex somatic symptom disorder, patients must complain of at least one somatic symptom that is distressing and/or disruptive of their daily lives. Also, patients must have at least two [Ed: now reduced to at least one since evaluation of the CSSD field trials] of the following emotional/cognitive/behavioral disturbances: high levels of health anxiety, disproportionate and persistent concerns about the medical seriousness of the symptom(s), and an excessive amount of time and energy devoted to the symptoms and health concerns. Finally, the symptoms and related concerns must have lasted for at least six months.”

“Future research will examine the epidemiology, clinical characteristics, or treatment of complex somatic symptom disorder as there is no published research on this diagnostic category.”

“…Just as for complex somatic symptom disorder, there is no published research on the epidemiology, clinical characteristics, or treatment of simple somatic symptom disorder.”

Source: Woolfolk RL, Allen LA. Cognitive Behavioral Therapy for Somatoform Disorders. Standard and Innovative Strategies in Cognitive Behavior Therapy.

Where are the professionals?

During the second public review, the Somatic Symptom Disorders proposals attracted more responses than almost any other category. The SSD Work Group is aware that patients, caregivers and patient advocacy organizations have considerable concerns. But are medical and allied health professionals scrutinizing these proposals?

This is the last opportunity to submit feedback. Psychiatric and non psychiatric clinicians, primary care practitioners and specialists, allied health professionals, psychologists, counselors, social workers, lawyers, patient advocacy organizations – please look very hard at these proposals, consider their safety and the implications for an additional diagnosis of an SSD for all patient illness groups and weigh in with your comments by June 15.

Criteria and rationales for the third iteration of proposals for the DSM-5 Somatic Symptom Disorders categories can be found here on the DSM-5 Development site. [Update: Proposals were removed from the DSM-5 Development website on November 15, 2012.]


1 Levenson JL. The Somatoform Disorders: 6 Characters in Search of an Author. Psychiatr Clin North Am. 2011 Sep;34(3):515-24.

2 Dimsdale JE. Medically Unexplained Symptoms: A Treacherous Foundation for Somatoform Disorders? Psychiatr Clin North Am. 2011 Sep;34(3):511-3.

3 Dimsdale J, Creed F. DSM-V Workgroup on Somatic Symptom Disorders: the proposed diagnosis of somatic symptom disorders in DSM-V to replace somatoform disorders in DSM-IV – a preliminary report. J Psychosom Res 2009;66:473–6.

4 DSM-5 Somatic Symptom Disorders Work Group Disorder Descriptions and Justification of Criteria-Somatic Symptoms documents, published May 4, 2011 for the second DSM-5 stakeholder review.

(Caveat: for background to the SSD Work Group’s rationales only; proposals and criteria as set out in these documents have not been revised to reflect changes to revisions or reissued for the third review.)

    Disorder Descriptions   May 4, 2011

    Rationale/Validity Document   May 4, 2011

© Copyright 2015 Suzy Chapman

DSM-5 in New Scientist: Psychiatry’s new diagnostic bible is creating headaches for doctors and patients alike

DSM-5 in New Scientist: “Psychiatry’s new diagnostic bible is creating headaches for doctors and patients alike”

Post #171 Shortlink: http://wp.me/pKrrB-293

A reminder that this third and final stakeholder review and comment period is scheduled to close on June 15.

On May 17, APA added the following statement to the home page of the DSM-5 Development site.

APA Position Statement on DSM-5 Draft Diagnostic Criteria

The official position of the APA on draft DSM-5 diagnostic criteria is that they are not to be used for clinical or billing purposes under any circumstances. They are published on the http://www.dsm5.org Web site to obtain feedback on these preliminary DSM-5 Task Force proposals from mental health professionals, patients, and the general public. They have not received official reviews or approval by the APA Board of Trustees and will not be available for clinical use or billing purposes until May 2013.

Two articles in this week’s online and print editions of New Scientist.

The first report, by Peter Aldhous, quotes Allen Frances, MD, who had chaired the development of the DSM-IV; APA research director and DSM-5 Task Force Vice Chair, Darrel Regier, and Dr Dayle Jones who is tracking DSM-5 for the American Counseling Association, on DSM-5 field trial kappa results and the relegation of Attenuated psychosis syndrome and Mixed anxiety/depression to the DSM-5 appendix.

This article is behind a paywall or a subscription to the print edition.

New Scientist 19 May 2012

Page 6 print edition


Psychiatry’s new diagnostic bible is creating headaches for doctors and patients alike

Online title Trials highlight worrying flaws in psychiatry ‘bible’

Peter Aldhous

Diagnosis: uncertain

HOW reliable is reliable enough?

The second article, “OPINION ‘Label jars, not people”, by James Davies, is accessible on the New Scientist website without payment or print edition subscription.

New Scientist 19 May 2012

Page 7 print edition

OPINION | James Davies

James Davies is a senior lecturer in social anthropology and psychotherapy at the University of Roehampton, London

‘Label jars, not people’

“LABEL jars, not people” and “stop medicalising the normal symptoms of life” read placards, as hundreds of protesters – including former patients, academics and doctors – gathered to lobby the American Psychiatric Association’s (APA) annual meeting.

The demonstration aimed to highlight the harm the protesters believe psychiatry is perpetrating in the name of healing. One concern is that while psychiatric medications are more widely prescribed than almost any drugs in history, they often don’t work well and have debilitating side effects. Psychiatry also professes to respect human rights, while regularly treating people against their will. Finally, psychiatry keeps expanding its list of disorders without solid scientific justification…

Read full article

More Kappa data from DSM-5 field trials

More Kappa data from DSM-5 field trials

Post #167 Shortlink: http://wp.me/pKrrB-27D

Further data from the DSM-5 field trials results have been released in a report by Deborah Brauser for Medscape Medical News.

You can read Ms Brauser’s report from the American Psychiatric Association’s annual conference here, though you may need to register for the site:

Medscape Medical News > Psychiatry

DSM-5 Field Trials Generate Mixed Results

Deborah Brauser | May 8, 2012

…Members of the task force said they hope to publish the full results “within a month.” However, the third and final public comment period for the manual opened last week and ends on June 15. Although the entire period is 6 weeks long, the public may only have 2 weeks to comment after the publication of the field trials’ findings.

“No previous field trial had such a sophisticated design. And it has resulted in more statistically significant data for specific disorders,” said Dr. Regier.

The current DSM-5 field trials, as well as field trials for past manuals, use Kappa score as a statistical measure of criteria reliability. A Kappa score of 1.0 was considered perfect, a score of greater than .8 was considered almost perfect, a score of .6 to .8 was considered good to very good, a score of .4 to .6 was considered moderate, a score of .2 to .4 was considered fair and could be accepted, and a score of less than .2 was considered poor.

 At adult sites, schizophrenia was shown to have a pooled Kappa score of .46. However, that is down from the .76 and .81 Kappa scores found in the DSM-IV and DSM-III, respectively, and it is less than the .79 score found in the International Classification of Diseases, Tenth Revision (ICD-10).

“It’s important to realize in some ways that the Kappa in the current field trial was from a totally different design…,” said Dr. Regier

Full report

This table has some of the results:

Reconstructed from data published by A Frances, DSM 5 in Distress, Psychology Today, 05.06.12


1 Boring Old Man has updated an earlier table here on his blog which incorporates additional data from the Medscape report: 

updated table
1 Boring Old Man | May 9, 2012

There are further, detailed commentaries from 1 boring old man on the DSM-5 field trial results and Kappa values here:

major depressive disorder κ=0.30?…     May 6, 2012

a fork in the road…     May 7, 2012

Village Consumed by Deadly Storm…     May 8, 2012

box scores and kappa…     May 8, 2012

Included in Ms Brauser’s report are data for “Complex somatic disorder”:

The field trials for the new proposed category Complex Somatic Symptom Disorder (CSSD) were held at Mayo. According to one of several tables within Ms Brauser’s report, the following data have been released for “Complex somatic disorder” [sic]:

Extract from DSM-5 Field Trials Generate Mixed Results, Deborah Brauser,  May 8, 2012

Disorder DSM-5 (95% CI) DSM-IV ICD-10 DSM-III
Major neurocognitive disorder .78 (.68 – .87) .66 .91
ASD .69 (.58 – .80) .59 – .85 .77 -.01
PTSD .67 (.59 – .74) .59 .76 .55*
Child ADHD .61 (.51 – .72) .59 .85 .50
Complex somatic disorder .60 (.41 – .78) .45 .42

CI, confidence interval; ASD, autism spectrum disorder; PTSD, posttraumatic stress disorder; ADHD, attention-deficit/hyperactivity disorder

*From the DSM-III-R.

CSSD is a new category for DSM-5 which redefines and replaces some, but not all of the existing DSM-IVSomatoform Disorders categories under a new rubric with a new definition and criteria.

It’s a mashup of the existing categories:

Somatization Disorder
Undifferentiated Somatoform Disorder
Pain Disorder

Following evaluation of the field trials, this new category, Complex Somatic Symptom Disorder is now proposed to drop the “Complex” descriptor, be named Somatic Symptom Disorder and absorb Simple Somatic Symptom Disorder (SSSD) – a separate diagnosis that had been introduced for the second draft, with criteria requiring fewer symptoms than for a diagnosis of CSSD and shorter chronicity.

In order to accommodate SSSD, criteria and Severity Specifiers for CSSD have been modified since the second draft. (More on this in the next post.)

Since CSSS (or SSD, as is now proposed) did not exist as a category in DSM-IV, or in ICD-10 or DSM-III, it’s unclear and unexplained by the table what data for which existing somatoform disorders have been used for Kappa comparison for this new category with data for ICD-10 and DSM-III, and how meaningful comparison between them would be.

You can find out more about how the field trials were conducted on the DSM-5 Development site.


Delay in publication of field trial results and no key documents in support of proposals

Stakeholders may not get to scrutinise a report on the field trials until as late as a couple of weeks before the public comment period closes.

There are no Disorder Descriptions and Rationale/Validity Propositions PDF documents that expand on category descriptions and rationales (at least not for the Somatic Symptom Disorders) and reflect revisions to proposals between the release of the second and third draft.

Yesterday, I contacted APA’s Communications and Media Office to enquire whether the Somatic Symptom Disorders work group intends to publish either a Disorder Descriptions or Rationale/Validity Propositions document, or both, to accompany this latest draft during the life of the stakeholder review period or whether these key documents are being dispensed with for the third draft.

I’ll update if and when APA Media and Communications provides clarification.


Related post:

Make Yourself Heard! says DSM-5’s Kupfer – but are they listening?

Make Yourself Heard! says DSM-5’s Kupfer – but are they listening?

Make Yourself Heard! says DSM-5’s Kupfer – but are they listening?

Post #166: Shortlink: http://wp.me/pKrrB-26L

Four further commentaries from 1 boring old man on DSM-5 field trial results and Kappa values:

major depressive disorder κ=0.30?…

May 6, 2012

a fork in the road…

May 7, 2012

Village Consumed by Deadly Storm…

May 8, 2012

box scores and kappa…

May 8, 2012

MedPage Today

Most DSM-5 Revisions Pass Field Trials

John Gever, Senior Editor | May 07, 2012

“…Darrel Regier, MD, the APA’s research director, explained that the trials were intended primarily to establish reliability – that different clinicians using the diagnostic criteria set forth in the proposed revisions would reach the same diagnosis for a given patient. The key reliability measure used in the academic center trials was the so-called intraclass kappa statistic, based on concordance of the “test-retest” results for each patient. It’s calculated from a complicated formula, but the essence is that a kappa value of 0.6 to 0.8 is considered excellent, 0.4 to 0.6 is good, and 0.2 to o.4 “may be acceptable.” Scores below 0.2 are flatly unacceptable.

Kappa values for the dozens of new and revised diagnoses tested ranged from near zero to 0.78. For most common disorders, kappa values from tests conducted in the academic centers were in the “good” range:

Bipolar disorder type I: 0.54
Schizophrenia: 0.46
Schizoaffective disorder: 0.50
Mild traumatic brain injury: 0.46
Borderline personality disorder: 0.58

In the “excellent” range were autism spectrum disorder [0.69], PTSD [0.67], ADHD [0.61], and the top prizewinner, major neurocognitive disorder [better known as dementia], at 0.78. But some fared less well. Criteria for generalized anxiety disorder, for example, came in with a kappa of 0.20. Major depressive disorder in children had a kappa value of 0.29. A major surprise was the 0.32 kappa value for major depressive disorder. The criteria were virtually unchanged from the version in DSM-IV, the current version, which also underwent field trials before they were published in 1994. The kappa value in those trials was 0.59.

But a comparison is not valid, Regier told MedPage Today…”

Read full report

DSM5 in Distress
The DSM’s impact on mental health practice and research.

Newsflash From APA Meeting: DSM 5 Has Flunked its Reliability Tests
Needs To Be Kept Back For Another Year

Allen J. Frances, M.D. | May 6, 2012

“…The results of the DSM 5 field trials are a disgrace to the field. For context, in previous DSM’s, a diagnosis had to have a kappa reliability of about 0.6 or above to be considered acceptable. A reliability of .2-4 has always been considered completely unacceptable, not much above chance agreement…”

Reconstructed from data published by A Frances, DSM 5 in Distress, Psychology Today, 05.06.12

“…No predetermined publication date justifies business as usual in the face of these terrible Field Trial results (which are even more striking since they were obtained in academic settings with trained and skilled interviewers, highly selected patients, and no time pressure. The results in real world settings would be much lower). Reliability this low for so many diagnoses gravely undermines the credibility of DSM 5 as a basis for administrative coding, treatment selection, and clinical research…”

Read full commentary

Scientific American

Field Tests for Revised Psychiatric Guide Reveal Reliability Problems for Two Major Diagnoses

Ferris Jabr | May 6, 2012

“…The kappa for generalized anxiety disorder was about 0.2 and the kappa for major depressive disorder was about 0.3.

“…These numbers are way too low according to the APA’s own scales—and they are much lower than kappas for the disorders in previous versions of the DSM. Regier and other members of the APA emphasized that field trial methodology for the latest edition is far more rigorous than in the past and that kappas for many diagnoses in earlier editions of the DSM were likely inflated. But that doesn’t change the fact that the APA has a problem on its hands: its own data suggests that some of the updated definitions are so flawed that only a minority of psychiatrists reach the same conclusions when using them on the same patient. And the APA has limited time to do something about it…”

“…Until the APA officially publishes the results of the field trials, nobody outside the association can complete a proper analysis. What I have seen so far has convinced me that the association should anticipate even stronger criticism than it has already weathered. In fairness, the APA has made changes to the drafts of the DSM-5 based on earlier critiques. But the drafts are only open to comment for another six weeks. And so far no one outside the APA has had access to the field trial data, which I have no doubt many researchers will seize and scour. I only hope that the flaws they uncover will make the APA look again—and look closer…”

Read full report

Psychiatric News | May 04, 2012
Volume 47 Number 9 page 1a-28
American Psychiatric Association
Professional News

DSM Field Trials Providing Ample Critical Data

David J. Kupfer, M.D.

This article is part a series of commentaries by the chair of the DSM-5 Task Force, which is overseeing the manual’s development. The series will continue until the release of DSM-5 in May 2013.

As of this month, the 12-month countdown to the release of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) officially begins. While the developers of DSM-5 will continue to face several deadlines over the coming year, the progress that has been made since APA’s 2011 annual meeting has been nothing short of remarkable.

One of the most notable and talked-about recent activities of the DSM revision concerns the implementation and conclusion of the DSM-5 Field Trials, which were designed to study proposed changes to the manual…

Read on

From the same article and note that

“After the comment period closes, visitors will no longer be able to submit feedback through the site, and the site will not reflect any further revisions to the draft manual in anticipation of its publication in May 2013. However, the site will remain live and viewable.”

Make Yourself Heard!

The DSM-5 Web site (www.dsm5.org) is open to a third and final round of feedback. For six weeks, patients and their loved ones, members of the profession, and the general public can submit questions and comments via the Web site. All will be read by members of the appropriate DSM-5 work groups.

A summary of changes made to the draft diagnostic criteria since the last comment period (May-July 2011) will help guide readers to important areas for review, but visitors are encouraged to comment on any aspect of DSM-5. After the comment period closes, visitors will no longer be able to submit feedback through the site, and the site will not reflect any further revisions to the draft manual in anticipation of its publication in May 2013. However, the site will remain live and viewable.

Psychiatrists can use this important opportunity to express their opinions about proposed changes and how they may impact patient care. Since http://www.dsm5.org was first launched in February 2010, the work groups have discussed— and in many cases, implemented draft changes in response to—the feedback received from the site. This final comment period presents a historic opportunity for APA members to take part in the DSM-5 revision process and help impact the way in which psychiatric disorders are diagnosed and classified in the future.

David J. Kupfer, M.D., is chair of the DSM-5 Task Force and a professor of psychiatry at the University of Pittsburgh Medical Center and Western Psychiatric Institute and Clinic.

Commentary on Dr Kupfer’s report from 1 boring old man


I boring old man | May 6,  2012

Further commentary from 1 boring old man on DSM-5 controversy

not a good time…

1 boring old man | May 5, 2012

Two commentaries from Allen Frances in response to APA field trial documents

Two commentaries from Allen Frances in response to APA field trial documents

Shortlink Post #130: http://wp.me/pKrrB-1GX

Allen Frances, MD, chaired the DSM-IV Task Force and a former chair of the Department of Psychiatry at Duke University School of Medicine, Durham, NC. He is currently professor emeritus at Duke.


References and resources

DSM-5 document: Q & A on DSM-5 Prevalence and Reliability January 12, 2012

DSM-5 document: Reliability and Prevalence in the DSM-5 Field Trials January 12, 2012

based on the The American Journal of Psychiatry article DSM-5: How Reliable Is Reliable Enough? Helena Chmura Kraemer, Ph.D.; David J. Kupfer, M.D.; Diana E. Clarke, Ph.D.; William E. Narrow, M.D., M.P.H.; Darrel A. Regier, M.D., M.P.H. January 01, 2012, Vol. 169. No. 1

DSM-5 Field Trials page

Consumer-Friendly Frequently Asked Questions about DSM-5 Field Trials
Frequently Asked Questions about DSM-5 Field Trials in Large, Academic Settings
DSM-5 Field Trial Protocol for Large, Academic Settings
DSM-5 Field Trial Protocol for Routine Clinical Practice Settings
APA’s Request for Proposals for Potential Field Trial Sites

DSM-5 Field Trials in Routine Clinical Practice Settings
Supplemental Material for Clinician Application to Own Institutional Review Board (IRB)

Inside DSM-5 Field Trials, Flyer, American Psychiatric Association Practice Research Network, December 2011

Commentary: DSM-5 Disorganization, Disarray, and Delays, Dr Dayle Jones, PhD, January 3, 2012


Two commentaries from Allen Frances, MD

Two Fallacies Invalidate the DSM-5 Field Trials
APA telegraphs that DSM 5 will be unreliable.

Allen Frances, MD | January 16, 2012

The designer of the DSM-5 Field Trials has just written a telling commentary in the American Journal of Psychiatry. She makes two very basic errors that reveal the fundamental worthlessness of these field trials and their inability to provide any information that will be useful for DSM-5 decision making.

1) The commentary states: “A realistic goal is a kappa between 0.4 and 0.6, while a kappa between 0.2 and 0.4 would be acceptable.” This is simply incorrect and flies in the face of all traditional standards of what is considered ‘acceptable’ diagnostic agreement among clinicians. Clearly, the commentary is attempting to greatly lower our expectations about the levels of reliability that were achieved in the field trials – to soften us up to the likely bad news that the DSM-5 proposals are unreliable. Unable to clear the historic bar of reasonable reliability, it appears that DSM-5 is choosing to drastically lower that bar – what was previously seen as clearly unacceptable is now being accepted.

Kappa is a statistic that measures agreement among raters, corrected for chance agreement. Historically, kappas above 0.8 are considered good, above 0.6 fair, and under 0.6 poor. Before this AJP commentary, no one has ever felt comfortable endorsing kappas so low as 0.2-0.4. As a comparison, the personality section in DSM III was widely derided when its kappas were around 0.5. A kappa between 0.2-0.4 comes dangerously close to no agreement. ‘Accepting’ such low levels is a blatant fudge factor – lowering standards in this drastic way cheapens the currency of diagnosis and defeats the whole purpose of providing diagnostic criteria.

Why does this matter? Good reliability does not guarantee validity or utility – human beings often agree very well on things that are dead wrong. But poor reliability is a certain sign of very deep trouble. If mental health clinicians cannot agree on a diagnosis, it is essentially worthless. The low reliability of DSM-5 presaged in the AJP commentary confirms fears that its criteria sets are so ambiguously written and difficult to interpret that they will be a serious obstacle to clinical practice and research. We will be returning to the wild west of idiosyncratic diagnostic practice that was the bane of psychiatry before DSM III.

2) The commentary also states: “one contentious issue is whether it is important that the prevalence for diagnoses based on proposed criteria for DSM-5 match the prevalence for the corresponding DSM-IV diagnoses” …. “to require that the prevalence remain unchanged is to require that any existing difference between true and DSM-IV prevalence be reproduced in DSM-5. Any effort to improve the sensitivity of DSM-IV criteria will result in higher prevalence rates, and any effort to improve the specificity of DSM-IV criteria will result in lower prevalence rates. Thus, there are no specific expectations about the prevalence of disorders in DSM-5.”

This is also a fudge. For completely unexplained and puzzling reasons, the DSM-5 field trials failed to measure the impact of its proposals on rates of disorder. These quotes in the commentary are an attempt to justify this fatal flaw in design. The contention is that we have no way of knowing what true rates of a given diagnosis should be – so why bother to measure what will be the likely impact on rates of the DSM-5 proposals. If rates double under DSM-5, the assumption will be that it is picking up previous false negatives with no need to worry about the risks of creating an army of new false positives.

This is irresponsible for two reasons. First off, we are already suffering from serious diagnostic inflation. Rates of psychiatric disorder are already sky high (25% in the general population in any year; 50% lifetime) and we recently have experienced three runaway false epidemics of childhood disorders in the past 15 years. Second, drug company marketing has been so abusive as to warrant enormous fines and so successful as to result in widespread misuse of medication for very questionable indications. Recent CDC data suggest that the severely ill remain very undertreated, but that the mildly ill or not ill at all have become massively overtreated, especially by primary care physicians.

The DSM-5 proposals will uniformly increase rates, sometimes dramatically. Not to have measured by how much is unfathomable and irresponsible. The new diagnoses suggested for DSM-5 will (mis)label people at the very populous boundary with normality. Mixed anxiety depression and binge eating disorder will likely have astounding high rates between 5-10% – that’s tens of millions people now considered ‘normal’ suddenly converted into mentally ill by arbitrary DSM-5 fiat. Psychosis risk and disruptive mood disorder will be extremely common in the young; minor neurocognitive among the elderly. Legions of the recently bereaved will be misdiagnosed as clinically depressed; rates of generalized anxiety and addiction will mushroom; and ADD which has already almost tripled will find even more room at the top. The field trial developers seem either unaware or insensitive to the unacceptable risks involved in creating large numbers of false positive, pseudo-patients.

Indeed, quite contrary to the blithe assertions put forward in the commentary, we should have rigorous expectations about prevalence changes triggered by any DSM revision. Rates should not be wildly different for the same disorder UNLESS there is clear evidence of a serious false negative problem and firm protections against creating a massive false positive problem. And new disorders with high prevalences should not be included without substantial scientific evidence and convincing proof of accuracy, reliability, and safety. We have known since they were first posted that none of the DSM-5 proposals comes remotely close to meeting a minimal standard for accuracy and safety. And now, the AJP commentary seems to be softening us up for the bad news that their reliability is also lousy.

The workers on DSM-5 ignore the often dire implications of drastically raising the prevalence of an existing disorder or adding an untested new disorder with high prevalence – i.e., the misguided and potentially harmful treatment, the unnecessary stigma, and rising health care costs that also cause a misallocation of very scarce resources. Just two examples. Do we really want even more antipsychotic medications prescribed for children, the elderly, and returning war veterans when these are already being used so loosely and inappropriately? Isn’t the current legal and illegal overuse of stimulant medications already a big enough problem without introducing a drastically lowered set of criteria for diagnosing ADD? Sad to say, DSM-5 has failed to do an adequate risk/benefit analysis on any of its suggestions. Every one of its changes is designed to chase elusive false negatives; none protects the interests of mislabeled false positives.

Given our country’s current binge of loose diagnostic and medication practice (particularly by the primary care physicians who do most of the prescribing), DSM-5 should not be in the business of casually raising rates and offering inviting new targets for aggressive drug marketing. Instead, DSM-5 should be working in the opposite direction – taking steps to increase the precision and specificity of its diagnostic criteria. And the texts describing each disorder should contain a new section warning about the risks of overdiagnosis and ways of avoiding it. It is impossible to say what is the “right” prevalence of any disorder, but it is careless and reckless to so dramatically increase the prevalences of mental disorders without evidence of need or proof of safety.

The DSM-5 field trials have cost APA at least $3 million (perhaps a whole lot more). They started off on the wrong foot by asking the wrong question – focusing only on reliability and completely ignoring prevalence. The deadlines for starting the trials and for delivering results have been repeatedly postponed because of poor planning, an excessively cumbersome design, and disorganized implementation. The results will be arriving at the very last minute when decisions should have already be made. And now we get a broad hint that the reliabilities, when they are finally reported, will be disastrously low.

What should be done now as DSM-5 enters its depressing endgame? There really is no rational choice except to drop the many unsupportable DSM-5 proposals and to dramatically improve the imprecise writing that plagues most of the DSM-5 criteria sets.

DSM-5: How Reliable Is Reliable Enough?
DSM 5 is willing to except poor quality.

Allen Frances, MD | January 18, 2012

This is the title of a disturbing commentary written by the leaders of the DSM 5 Task Force and published in this month’s American Journal of Psychiatry. The contents suggest that we must lower our expectations and be satisfied with levels of unreliability in DSM 5 that historically have been clearly unacceptable. Two approaches are possible when the DSM 5 field trials reveal low reliability for a given suggestion: 1) admit that the suggestion was a bad idea or that it is written so ambiguously as to be unusable in clinical practice, research, and forensics; Or, 2) declare by arbitrary fiat that the low reliability is indeed now to be relabeled ‘acceptable’.

In the past, ‘acceptable’ meant kappas of 0.6 or above. When the personality disorders in DSM III came in at 0.54, they were roundly derided and given only a reluctant bye. For DSM 5, ‘acceptable’ reliability has been reduced to a startling 0.2-0.4. This barely exceeds the level of agreement you might expect to get by pure chance.

Previously in its development, DSM 5 has placed great store in its field trials. This quote is from the Chair of the DSM 5 Task Force: “There’s a myth that all the decisions have been made, when in fact, all the decisions haven’t been made. Just because things have been proposed doesn’t necessarily mean they’ll end up in the DSM-5. If they don’t achieve a level of reliability, clinician acceptability, and utility, it’s unlikely they’ll go forward.”

And this quote is from a 2010 interview given to a science writer by the head of the DSM 5 Oversight Committee: “It’s going to be based on the work of the field trials – based on the assessment and analysis of them. I don’t think anyone is going to say we’ve got to go forward if we get crappy results.”

The DSM 5 tune has now changed dramatically. The commentary written for AJP by the leadership of DSM 5 Task Force appears to be suggesting that they will, in fact, “go forward,” and with sub par reliabilities of 0.2-0.4. Now consider that the original field trial plan was to have a second phase to permit fixing those diagnostic criteria that were found to have unacceptable reliability in the first phase. These would go back to the workgroups who could then rewrite the offending criteria and retest the new version in the second phase of the field trial. But poor planning and administrative foul-ups kept pushing back the field trials so that they are now at least 18 months late in completion. As time was running out, DSM 5 leadership quietly dropped the second phase of the field trials, removing any reference to it from the timeline posted on the DSM-5 website. Their Plan B substitute for adequate field testing appears in AJP- To wit: a drastic lowering of the bar for what is ‘acceptable’ reliability.

Can ‘accepting’ unacceptably poor agreement uphold the integrity of psychiatric diagnosis? Poor reliability degrades our ability to communicate with one another clinically, and prohibits meaningful research. ‘Accepting’ as reliable kappas of 0.2-0.4 is to go backwards more than thirty years to the days of DSM II. Before DSM III, Bob Spitzer and Mel Sabshin saw the need to develop a criterion based system that could achieve reasonable diagnostic agreement. This is the very minimum condition necessary for current clinical work and future progress in psychiatry.

%d bloggers like this: