― 1 ―
Since one cannot, legally or morally, as a true experimental design would have it, select a number of individuals, divide them into two groups and provide services to one and withhold them from the other, the question looms: where do we get a control group? It would not be possible to find a control group whose characteristics and contextual setting is the same as a randomly selected group of DR clients. How could such individuals be found? If indeed they were found, does one somehow clandestinely monitor their progress over time to compare it with DR clients?
There is perhaps one feasible method of performing an evaluation which has somewhat of an experimental nature. Of course, the main feature of experimental design we are interested in capturing is that of comparing a group who received the services of DR with a group that did not. One way to find two such groups would be to seek out a particular class of disabled people who have in common an encounter with some public or private agency. One example would be a rehabilitation hospital. A search of the records of several rehab hospitals could generate a list of individuals with similar characteristics who passed through the hospital several years ago. These people could be contacted (although a good many will not be found) and their progress evaluated with respect to finding employment. The experimental group would be those served by DR. The control group would be those who have never had any contact with DR. Information on the services provided to those served by DR could be gathered from DR files. Information on the status of certain
― 2 ―measures or indicators tied to DR's goals on those not served by DR would rely on their memory of their situation several years ago, whatever time were selected as the beginning of the experimental program. In addition to measures tied to DR's objective function (preparing clients for employment) measures of a person's improvement in ability to live independently and perform activities of daily living could be chosen and used to evaluate progress of both groups in this area.
There are several disadvantages to an evaluation design outlined above. The primary limitation this design has in terms of shedding light on the central issues of DR is that it evaluates the effect of the program on only one class of client, i.e., orthopedically disabled. Extrapolating the results from this class of clients to DR's entire caseload would not be appropriate.
Another limitation of the above design in terms of scientific method is the fact that there is no actual "before" period to gather data. The above design relies on the subject's memory and his candidness. This can impose considerable uncertainty to the findings.
One additional problem presented by the above design is the notion that circumstances which kept an individual from coming into contact with DR may also have retarded their progress along whatever measures are chosen. If such were the case there would be considerable bias present. Such a bias cannot be dealt with unless it can somehow be measured; which is unlikely.