Complex regional pain syndrome type I (CRPS-I), formerly reflex sympathetic dystrophy (RSD), is a chronic pain syndrome of unknown aetiology. Its diagnosis is a clinical one, for which several criteria systems have been defined. Despite their widespread use, the reliability of these criteria has never been studied. In this interobserver study 25 chronic CRPS patients were interviewed and examined by six physicians. Through structured questionnaires signs, symptoms, and diagnosis were recorded, after which observer agreement for these was calculated with κ statistics. Physicians' agreement in assessment of signs and symptoms in CRPS patients varied greatly. More importantly, final diagnosis of CRPS showed poor observer agreement (κ: 0.20). The κ values were higher, had physicians applied IASP criteria, but still insufficient. The application of Bruehl's criteria results in a fair κ of 0.38, but then frequency of CRPS diagnosis in our study population decreased from 73% to 43% in comparison with physicians' own diagnosis. We conclude that, using current criteria systems, the diagnosis of CRPS is not reliable.