Ask the Scholar
Document scope · 1 page
Scholar
Ask about this object, its catalog metadata, its source description, or the page inventory.
For page-specific OCR and visual context, open one of the page chats.
Scholar Source Context
Document identity
localId
351165416
label
"Functional Limitations Concepts in Vocational Rehabilitation: A Delphi Study" [1978]
core
doc
dtoType
document
citationUrl
pageCount
1
Source metadata
id
351165416
contentType
document
title
"Functional Limitations Concepts in Vocational Rehabilitation: A Delphi Study" [1978]
citationUrl
collections
Lex Frieden Collection: Records on Disability Rights
Printed Materials
imageCount
1
hasImages
yes
source
import
hasTranscription
no
Source extras
naId
351165416
levelOfDescription
fileUnit
recordType
description
ocrSource
nara-archive
Single page context
seq
1
pageIndex
0
type
document
mediaId
92ce4e2e5c888dde
ocrText
Originally Processed With FOIA(s):
FOIA Number:
S
S
FOIA
MARKER
This is not a textual record. This is used as an
administrative marker by the George Bush Presidential
Library Staff.
Record Group/Collection: Donated Historical Materials
Collection/Office of Origin: Frieden, Lex, Collection
Series:
Printed Materials
Subseries:
Papers/Books
OA/ID Number:
52114
Folder ID Number:
52114-002
Folder Title:
"Functional Limitations Concepts in Vocational Rehabilitation: A Delphi Study" [1978]
Stack:
Row:
Section:
Shelf:
Position:
Lex,
Hereare the Delphi
results. I'll send you a
final report of our study
as soon as t becomes available
Tim
FUNCTIONAL LIMITATIONS CONCEPTS
IN VOCATIONAL REHABILITATION
A DELPHI STUDY
Timothy C. Muzzio
Christine Tuve Burris
Senior Research Associate
Research Assistant
indices, inc.
3110 Columbia Pike
Arlington, Virginia 22204
September, 1978
PREFACE
The following report presents the actual results from each round of a Delphi
Study conducted on the topic of functional limitations concepts in vocational
rehabilitation. These results will be of interest to those rehabilitation
professionals actively involved in developing the concept of functional
limitations. A more useful document to others interested in this area would
be "Functional Limitations State-of-the-Art Review", of which this report
is an appendix. The State-of-the-Art Review summarizes the Delphi results
and presents additional information on the topic of functional limitations.
The present report presents the actual results of the Delphi Study, including
verbatim comments made by Delphi participants. It should be used in con-
junction with the previously cited report, for the purpose of identifying
issues and factors to be addressed or taken into account when developing a
scale related to the area of functional limitations.
There were three rounds to this Delphi Study. The results of each of these
rounds is separated by yellow pages in the present report. Comments are given
for each individual round. Therefore, to get a feel for the concerns of the
participants on a particular topic, all rounds should be read. Additionally,
the first, round also contains the "information for respondents". This presents
a description of the Delphi process, definitions, and the rating.system. This
should be read carefully prior to reading the Delphi results.
Generally the format for each round of the Delphi results consists of the left-
hand page, and right-hand page. The left-hand page presents the questionnaire
as it was provided to Delphi participants for their responses. It should be
noted that on the third round, some boxes on the left-hand page are marked with
initials. These initials represent the consensus reached on the items they
are associated with. "I" means the item was found to be important; "D" desir-
able; "F" feasible; "A" adequate; "P" probable; and, "L" likely. An "N" prior
to one of these initials should be read as "not (desirable, important, or what-
ever is appropriate).
The right-hand page presents the results. The box in the top left-hand corner
summarizes the numerical results of ratings on items on the opposite page.
The remainder of the page presents all comments received, in verbatim form.
Note: The results of the first round are not properly coordinated. That is,
comments (right-hand pages) are not always presented opposite to their
appropriate page. Page numbers should be matched by the reader. The
second and third rounds are accurate. We are sorry for this inconven-
ience.
DELPHI PARTICIPANTS
Delphi Panelists
Alfred Butler, University of Wisconsin
Nancy Crewe, University of Minnesota
Margaret Brown, Institute of Rehabilitation Medicine
Susan Stoddard, Berkeley Planning Associates
Richard Melia, Rehabilitation Services Administration
Robert Struthers, Michigan Bureau of Rehabilitation
Stephan Harasymiw, Rehabilitation Institute of Chicago
John DeWilde, Department of Health, Education and Welfare
Donald Thurman, Arkansas Department of Rehabilitation Services
Design Team
Timothy C! Muzzio, indices, inc.
Christine Tuve Burris, indices, inc.
Joseph LaRocca, Rehabilitation Consultant
-----
FUNCTIONAL LIMITATIONS
STATE-OF-THE-ART REVIEW PROJECT
A Delphi Study
Round 1,
May 1978
INFORMATION FOR THE RESPONDENTS
You are about to participate in a new approach to issue analyses -- a
Delphi exercise. In this exercise, a series of questionnaires are given
to a group of respondents to determine the group's views on a particular
topic. In this instance, the topic is Functional Limitations and what
should or could be done in the area.
You should discard all traditional theories of surveys and questionnaires
when approaching this Delphi exercise. We are not designing a short answer,
"quickie" questionnaire and our respondent group is not a random sample
designed to reflect the population as a whole. Our respondent group is
small, certainly not random, and in some ways non-representative of the
general population. We are interested in involving our respondents, not
only in thinking deeply about the questions we ask, but in formulating
new questions and exploring new alternatives. We are asking you as a
respondent, to spend three one-half to one day time periods at approxi-
mately three week intervals.
The Delphi approach is cumulative. The first round is largely exploratory
and designed to open up new areas of thought. In the second round the areas
of interest are narrowed and group views on certain topics begin to emerge.
By the third round, there is a clear indication of the group's opinions and
attitudes on the subject of the exercise, and some discussions and decisions
may be carried out, based upon the group's concensus or polarization. On each
round, the reasoning of participants is fed back to all respondents for their
appraisal. It should be emphasized that also unlike the traditional one-
shot survey technique, we are dealing largely with factors nonquantifiable
by analysis.
You will notice that the pages are arranged so that comments can be written
on the page opposite the question. Please feel free to use this space, and
additional pages, if necessary. All necessary steps will be taken to preserve
the anonymity of the respondents, this being an essential feature of this
technique.
Questionnaires in a Delphi exercise are only meant to be a stimulus and not
a straight jacket. As a respondent you have the following options with
respect to any particular question, sub-question or alternatives presented:
- You may choose not to answer a question if you feel your judge-
ment would be "RISKY." In this situation, if you feel the
judgement is an important one to be made, we would hope for a
comment on the type of information that would aid you or others
in making the particular judgement.
- You may choose to rewrite, as a comment, a particular question
and answer your version if you feel the original is misleading.
- You may suggest questions you would like to see in the next
questionnaire if you feel they would clarify an issue or raise
a new alternative that the group should consider.
2.
- You may consult with associates on specific items in the
questionnaire or utilize other information sources normally
available to you. However, you should respond as an indi-
vidual and not as a spokesman for a particular group.
- You may, and are encouraged to, express short arguments
or comments on any judgement about which you feel confident.
This may include items of information that lead you to the
judgement. The shorter the comment, the greater the proba-
bility it will be included as is in the summary which is fed
back to you on the next round.
Whenever a consensus of judgement is obtained on an issue, we will usually
drop it from further exploration in succeeding questionnaires. When a
polarization of views occurs, we will attempt to develop questions designed
to highlight reasons for the polarization. The degree to which the design
team chooses to explore a majority-minority type polarization will be a
stronger function of the arguments or comments made than of the actual number
of individuals taking a particular view.
SPECIFIC INSTRUCTIONS FOR THE RESPONDENTS
(1) You have been provided two copies of the Delphi questionnaire. Return
only one; the other should be kept handy for reference in succeeding
questionnaires. It would be helpful to us if you would have written
responses typed out.
(2) Return your response three days after the questionnaire reaches you.
(3) You should expect three questionnaires for the complete exercise. Hope-
fully, we will provide a new one with the summary within two weeks after
each response.
(4) Do not hesitate to contact by phone those of us on the design team if you
need clarification on the questions or instructions (Tim Muzzio or
Christine Burris -- (703) 521-1810).
The goal of the design team is to maintain complete impartiality in the
summary of the responses and the design of questions. Unintentional
violations of this principle may occasionally occur and we would appreciate
your comment when you find this to be the case.
E AND NE RESPONSES
We will normally consider all answers as reflecting the best judgement of a
person in a policy or decision position relying on his experience and back-
ground in such positions. We will not consider the answer to reflect expert
opinion on a particular topic unless the respondent wishes to designate his
answer as such. Therefore, if you feel your response on a particular topic
or issue is that of an unusually well-informed individual because of special
knowledge or background you may indicate this by placing the initial "E" (Expert)
conspicuously next to your answer. When an "E" response differs from the
average response of the group, the design team will pay special attention to
the issue. At the opposite end of the spectrum, a response of "NE" (Not an
Expert) will be given proper consideration also.
3.
EXPLANATION OF ALTERNATIVE DESCRIPTORS AND MODIFIERS
The following descriptors will be used throughout the exercise to provide
possible expressions of judgement. The respondents are asked to keep the
following guidelines in mind when responding to the tabulated questions or
using the descriptors in a comment. This is important in establishing
compatibility among the responses even though the definitions may not be
universally agreeable.
IMPORTANCE (Priority or relevance)
4 Very Important
a most relevant point
first order priority
has direct bearing on major issues
must be resolved, dealt with or treated
3 Important
is relevant to the issue
second order priority
significant impact but not until other
items are treated
does not have to be fully resolved
2 Slightly Important
insignificantly relevant
third order priority
has little importance
not a determining factor to major issue
1 Unimportant
no priority
no relevance
no measurable effect
should be dropped as an item to consider
CONFIDENCE
4 Certain
low risk of being wrong
decision based upon this will not be wrong
because of this "fact"
most inferences drawn from this will be true
3 Reliable
some risk of being wrong
willing to make a decision based on this
but recognizing some chance of error
some incorrect inferences can be drawn
2 Risky
substantial risk of being wrong
not willing to make a decision based on
this alone
many incorrect inferences can be drawn
1 Unreliable
great risk of being wrong
of no use as a decision basis
4.
DESIRABILITY (Effectiveness or benefits)
4 Very Desirable
will have a positive effect and little or
no negative effect
extremely beneficial
justifiable on its own merit
3 Desirable
will have a positive effect, negative
effects minor
beneficial
justifiable as a by-product or in
conjunction with other items
2 Undesirable
will have a negative effect
harmful
may be justified only as a by-product of a
very desirable item, not justified as a
by-product of a desirable item
1 Very Undesirable
will have a major negative effect
extremely harmful
not justifiable
PROBABILITY (Likelihood)
4 Very Probable
almost certain to occur
strong indications of this happening
3 Probable
better than a fifty-fifty chance of occurring
some indications of this happening
2 Improbable
less than a fifty-fifty chance of occurring
some indications of this not happening
1 Very Improbable
almost certain not to occur
strong indications against this happening
5.
FEASIBILITY (Practicality)
4 Definitely Feasible
no hindrance to implementation
no R & D required
no political roadblocks
acceptable. to the public
3 Possibly Feasible
some indication this is implementable
some R & D still required
further consideration or preparation to be
given to political or public reaction
2 Possibly Infeasible
some indication this is unworkable
significant unanswered questions
1 Definitely Infeasible
all indications are negative
unworkable
cannot be implemented
ADEQUACY (Sufficiency)
4 Very Adequate
Area/subject has been thoroughly studied,
abundant information available
Results sufficient enough to be utilized
immediately
3 Adequate
Sufficient information available, but some
details need to be claried/studied
Results could be utilized with a little
additional information
2 Inadequate
insufficient information on this subject is
available, but efforts have been started
substantial effort needed before utilization
of results is possible
1 Very Inadequate
Area/subject hardly studied at all
Very little, if any, information available
6.
I. DEFINITIONS
This section of the study is included to provide a common basis of
understanding for all participants. Although you may not agree with
the definitions, use them as operational definitions for the purposes
of this study.
The following words are not alphabetically arranged. Rather, they are
grouped into meaningful units to help the reader discriminate between
the meanings of closely related words. If you feel that other words
should be included here, please make note of those words.
*1 Impairment - a physiological, anatomical, or mental loss or other
abnormality, or both
*1,2 Disability - an inability or limitation in performing roles and tasks
expected of an individual within a social environment
*1 Taken from: Whitten, E.B. (ed), Pathology, impairment, functional
limitation and disability - Implications for practice, research,
program and policy development and service delivery. Report of First
Mary E. Switzer Memorial Seminar, The National Rebabilitation Association,
Washington, D. C., 1975.
*2 It is not the purpose here to involve ourselves in the controversy sur-
rounding the definitional differences between the terms handicap and
disability. On this point we do not wish for you to comment as it
would divert attention from the main purpose of our study. For
purposes of this study, handicap and disability are synonomous terms.
A quotation from Whitten (1975) may best explain our reasoning.
"The term -handicap' has been widely used in the field of rehabilitation
in the way we are using the term disability. The term handicap emerged
during the early developments of rehabilitation counseling in an attempt
to distinguish "medical" entities which constitute the professional domain
of physicians from the "non-medical" entities which were to become the
domain of rehabilitation. The AMA's Committee on Medical Rating of
Physical Impairment, however, made it clear that disability is not a
medical phenomenon and therefore is not an exclusive area of physicians'
competence and practice. The term handicap has never been operationally
defined in a way that allows an assessment of what specifically it
designates. Until such definitions are available it will be difficult
to say whether what it designates is an exact equivalent of what is here
labeled disability. The use of the word handicap in so many fields that
have no bearing on impairments or rehabilitation can be used as argument
for dropping the use of this word as a label for functional limitation."
7.
Functional limitation - degree of inability or limitation in performing
a specific role or task which is expected of an individual within a social
environment
Functional Capacity - degree of ability to perform a specific role or task
which is expected of an individual within a social environment
Rehabilitation indicators - variables which can indicate some given end
result related to the rehabilitation process such as rehabilitation
potential, or employability
Severe handicap - disability which requires multiple services over time;
constitutes or results in a substantial handicap to participation in
society
Independent living - rehabilitation of a person with the purpose of encouraging
maximum participation in the social environment
Social environment - the milieu surrounding a person in society including
work, home, community, education, recreation and other activities
General Comments
-- I am in agreement with your position not to discuss the issue of
handicap versus disability. However, I think your definitions leave
no way to address the reality of social handicaps to obtaining employ-
ment -- being black, disfigured, poorly educated, a woman, etc. A
scale which includes such items may have more utility than a scale
which reflects only functional limitations which reside in the
individual. If there is confusion on this distinction among your
respondents, it may influence the results you obtain. My responses
are based on my evaluation of the utility of a scale measuring limi-
tations in performance by the individual only. It might have been
desirable to have Question III first to let respondents settle on
their own definition.
(Design team comment: As is stated here, this concern is dealt with
in Question III).
-- As a matter of general introduction, I would like to say that I think
in future years the only feasible way to establish the extent to which
a disabling condition or impairment presents a handicap to employment
or independent living is through a measure of the functional limitations
demonstrated by the individual. I have always had some difficulty in
accepting as a logical system the categorical designation of certain
groups of people as severely disabled simply because they happen to
have that disabling condition. Consequently, it is my personal belief
that the only feasible measure of an individual's dearee of impairment
and relating that to handicapping conditions for work or independent
living is through a measure of functional limitations based upon what
the individual is and is not able to do in relation to vocational
objectives.
-- 2 basic concerns overall to begin with
(i) use of term "functional limitations" while understandable is
unnecessarily negative in connotation. Would prefer term
"functional capacities", although same attributes would be
under consideration; measures should permit manifestations of
superior performance as well;
(Design team comment: See cover letter).
(ii) understand forced choice of ++, +, - and -- on 4 point scale.
However, some items could use a zero point.
(Design team comment: If you feel a zero is warranted, please use a
1 rating).
(Please write comments on opposite page
)
8.
II. BASIC ISSUES
A. User Groups/Needs
Functional limitations scales can be used by various groups. Each group
would use the scale(s) for different purposes. Scale developers must
tailor their scales to the most important user groups and their needs.
It is the purpose of this section to determine the priority user groups
and their specific needs. Please indicate what you would judge to be
the importance of functional limitation scales for each group identified
and the importance of the purposes listed for each group. Please also rate
the degree of confidence you have in your response.
For example, under the importance column, an appropriate response for a
rating of "4" in importance and a rating of "3" in confidence would appear
as 11
11
4
3
Importance/
Confidence
1. Clients
Please rate the importance of functional limitations
scales for the rehabilitation client and your confidence
in your response
Please rate the importance of the following possible
purposes of functional limiation scales for clients
and your confidence in your response
a) assessing self progress
b) to gain realistic understanding of the effects
of their disabilities
c) developing an IWRP with the Counselor
d) other (list and rate)
Question
Number
4
31
2.1
Comments Concerning Page S
I
4 2 I
1-9
1 3 2:1
-b 2.32
II-A-1
- C 3'22
-- I think the use of scales with most clients would be
-d*1
I
about like using IQ scores -- suggesting many questions
and not answering very many. Clients know when they
are progressing toward their goals without having an
artificial number applied to them.
-- Note: For many programmatic uses of such measures, clients will be unaware of
use. Also, clients for the most part will not be skilled in interpreting such
"scores." However, since scales such as this may be used to determine eligibility
for services and dollars, relevance to clients is very important (e.g., the
Comprehensive Needs Study demonstrations have used Barthel scores to determine
eligibility for the CNS programs.) This eligibility effect of functional limi-
tations scales far outweighs the items in "a" through "c" on the questions.
-- Clients - Vocational Rehabilitation is based on assumptions of joint involvement
of clients and counselors in identifying client needs, objectives, and ways to
achieve the desired outcomes. Since many clients will have only a partial, and
perhaps incorrect, knowledge of their abilities as well as strategies for achiev-
ing goals, rational services planning depends on "bridging the information gap
between counselor and client. While offering advantages of objectivity in
individual assessment, increased client awareness of possibilities for advance-
ment, efficiency in joint planning, and a plan by which decisions can be tracked,
functional limitations scales could present disadvantages as well such as ex-
pectations beyond available resources, unwarranted emphasis on limitations
(e.g., what the client can't do), and possible conflict between "objectivity"
of the scale criterion and client "faith" or "determination" or "intuition".
Regardless of what measure is used, the importance is the fact of judging.
And since individual assessment is a key determinant of what services are to
be provided for what intermediate objectives. and ultimate goal, this topic is
very important for clients.
-- If the instrument has proven effective clinically, the counselor and client
could complete ratings independently and compare the results -- a reality-
testing exercise.
-- One wouldhave to be extremely careful of language if the instrument is to be
used with clients. Avoid shortcuts and labels.
-- I am interpreting importance to user in terms of direct use. For example,
indirect use of scale data, i.e., via the counselor would be very important
(4), information given directly to client would be less important.
(Design team comment: You are correct in your understanding that we are
interested in direct use).
II-A-1-a
-- Under Item 1. on Page 8 the question arises assessing self-progress. I am
not sure this is a highly valuable instrument or procedure for assessing self-
progress. I think functional limitations measures should be more for the
benefit of the rehabilitation agencies who have the ultimate responsibility
for declaration of eligibility and provision of services. While I do not
object to this kind of information being available to clients or recipients
of services, I think its ultimate value lies in the technical information it
provides to the individuals serving the clients.
-- Would require too much counselor interpretation.
*1 - eligibility determination
(Please write comments on opposite page
)
9.
Importance/
Confidence
2. Counselors
Please rate the importance of functional limitations
scales for the rehabilitation counselor and your
confidence in your response
Please rate the importance of the following possible
purposes of functional limitation scales for counselors,
and your confidence in your response
a) eligibility determination
b) developing a rehabilitation plan
c) insuring effective delivery of services
d) assessing client progress
e) counselor performance appraisal
f) weighted case closure
g) minimize conflict in client assessment of services
h) job plaement activities
i) other (list and rate)
Comments Concerning Page 8 (b)
-- Many of the functional scales will measure change only in the early stages of
rehabilitation (crisis intervention and stabilization.) By the time a client
sees a VR counselor, little change on many of the scale items would be antici-
pated. And with progressive disability, how do you interpret "no change", or
slippage on a scale? It may be an erroneous assumption that such scales can
indicate "progress," at least some scales used now.
(Design team comment: This may be one thing to keep in mind when answering
the brevity/detail question on page 18).
-- This is confusing. Do you mean self-rating?
(Design team comment: We have modified this question).
-- Many dimensions are not subject to change, and therefore they would not be
useful for assessing progress. Some may be appropriate. It might be necessary
to flag general area and develop more specific, finely-graded behavioral items
to tap progress.
II-A-1-b
-- Make sure the instrument has been well validated before drawing conclusions
about the effects of impairments.
-- Potentially useful counseling tool for this purpose.
II-A-1-c
-- Can see a "profile" being helpful theoretically but can't "see" through the
process well enough to be confident.
*
1 - eligibility determination
Question
Importance
Number
4
31
2
1
Comments Concerning Page 9
2
6
2-a
6 /
II-A-2
-b -
4
I
2
-- Scale(s) would be extremely useful to counselor much
- -C
2
I
3
I
in excess of any other group.
- I
2 2 2
!
-- Overall vote for this, as in the previous section on
- e
1 3 I
2
clients, is 4/4. This is not a cumulative score, but
- f
I 3 3
based entirely on the eligibility question, which in
-9
I
4
my opinion outweighs in personal impact on counselor
and client any of the other concerns listed.
- h
2
3
)
I
-- Counselors - Functional limitations scales enhance the
-
/
professional counseling role. They comprise a specialized
1
body of knowledge which the professional counselor can
-
interpret, act upon, and utilize in provision of services.
I
-
Professional training and continuing education should be
added as a purpose since in learning to use the scales,
-
/
counselors will acquire professional skills in new disa-
bility-related areas.
(Design team comment: This is addressed in Question 6-a, page 13, which attained
a high degree of consensus).
Ineligibility determination should be added as a purpose because denial of entry
to a service must also be justified, as is eligibility. Nagi points out that
"false negatives" (saying no when you should have said yes) is potentially more
harmful than "false positives" (saying ok - eligible, to a non-eligible).
(Design team comment: To cover this we have rephrased item "a" to read "fair
and equitable eligibility determination").
-- I have some difficulty with the terminology of functional limitation scale. I
suppose an assumption was made that functional limitations will be measured on
the basis of some sort of a paper and pencil scale. Consequently, I find this
is a most disturbing approach and feel it should be left open and that it may
be necessary to have a combination of scales individualized interpretation,
something approaching the GULHEMP procedure, goal attainment scaling or any
kind of measuring devices along those lines. I think it would be a mistake
at the outset to assume that any kind of measure of functional limitations
will be based simply upon a scale. Scale for this purpose needs considerably
more definition than the terminology in question "2" would indicate.
(Design team comment: We agree and have changed "scale" to "instrument").
-- Any sort of clinical application must follow extensive research work.
-- May provide a means of more standardized eligibility determination.
-- May help provide a better way of organizing services, making sure all the bases
are touched.
II-A-2-a
-- Scales would be helpful to counselors for eligibility determination only if
supporting research and policy statements are based on the scales. They have
the potential to help the counselor decide if there is "reasonable expectation"
that the client will enter gainful employment. However, if administrators will
not make statements about "cut-off levels", etc. (and I don't think they will),
then they will be of little help.
-- "Eligibility" was the first point for counselors. Obviously it is at least
as important for clients.
*
1 - ineligibility determination
*
2 - professional training and continuing education
*
3 - caseload management
*
4 - framework for organizing a comprehensive initial interview
10.
Importance/
Confidence
3. State Agency Personnel
Please rate the importance of functional limitation
scales for the State agency personnel and your confidence
in your response. State agency personnel include the
State administrator, counselor, supervisor, programs
evaluator, etc., but in this instance does not include
the counselor
Please rate the importance of the following possible
purposes of functional limitation scales for State agency
personnel, and your confidence in your response
a) estimating the population at risk
b) setting long/short-range goals
c) program planning/evaluation
d) set priorities in program
e) budget justification
f) counselor performance appraisal
g) improved caseload management
h) 'other (list and rate)
Comments Concerning Page 9 (b)
II-A-2-b
-- Scales will help develop plans only if they include subscales which are some-
what diagnostic.
-- Developing a rehabilitation plan should say "with the client" to be parallel
with #1. Both Q. 1 and 2 should refer to an IWRP, which should be written
out -- Individualized Written Rehabilitation Program (SIC).
II-A-2-c
-- Thorough assessment may contribute to effective delivery of services, but
certainly will not "insure" it. Many additional elements go into total
provision of service.
-- This is too general -- Do you mean as a pre-post outcome measure use to assess
effectiveness of services?
-- Refers to services. A purpose should be included relating to intermediate
rehabilitation objectives -- "the steps which must be achieved before the
long-range vocational goal can be attained, i.e., those medical, social,
personal outcomes which must be attained and upon which the attainment of
the long-range goal is dependent.
-- May be useful for selected services, e.g., in training of specific skills,
where measure of limitation is reliable and objective of service is quite
specific. See less use for such services as psychotherapy.
II-A-2-d
-- It is unclear that client progress can be captured accurately by scales such
as the Barthel, after stabilization.
II-A-2-f
-- Will provide 1/2 of necessary information, other 1/2 includes judgement of
difficulty of specific limitations.
-- On weighted case closure: Little in the literature has established clear
relationship between "severity" (or functional limitation) and "difficulty."
While there is obviously some relationship, analysis is needed before any
scale should be recommended for a weighting scheme which is to adjust for
difficulty. (Consider the blind, by definition "severe," yet demonstrating
a vast range in terms of "difficulty").
II-A-2-q
-- Don't understand this one.
-- This item is not clear to me.
-- Absolutely unclear to me what this means
-- What does minimize conflict mean?
*
1 - ineligibility determination
*
2 - professional training and continuing education
*
3 - caseload management
*
4 - framework for organizing a comprehensive initial interview
Comments Concerning Page 9 (c)
II-A-2-h
-- In the current system, I would vote 1. Assuming that job placement activity
would include more information, employers cognizant of the meaning of scales,
etc., I would vote 2.
-- Important, but must be carefully used, since stress is on limitations not assets.
-- Important and useful contributor, but not sufficient in and of itself. A
F.L. measure is general and lacks much data regarding abilities, interests,
values, etc.
II-A-2-i
-- Case-load management can be improved by a more accurate assessment of distri-
bution of client characteristics and relating this to time and energies spent
in matching these to case service monies and community resources.
*
1 - ineligibility determination
*
2 - professional training and continuing education
*
3 - caseload management
*
4 - framework for organizing a comprehensive initial interview
Question Imoorta nce
Number 4 3' 2 1
Comments Concerning Page 10
3
151
3-a 1 22
II-A-3
-b
1
33
-- State Agency Personnel - Management has an indirect
-C -
4
3
relationship to functional limitations measures.
- d 232
Managers must look at possible dysfunctions associated
with the use of measures such as increased start-up
-e
43
costs, need for special staff to train and implement
scales, etc. Benefits may be less obvious the more
-+ -
I
4
I
/
distance from counselors/client. Hencé evaluation of
direct client outcomes may be greatly impacted, case-
-g 313
load management affected, and staff specialists and
providers impacted. But budget processes, planning,
*44-
-
I
priority setting, and estimating population at risk are
activities that managers must do working with other
-
/
policy matters. Distance from specialized knowledge
of disability measures dilutes the value of such data
-
I
for these purposes.
-- Functional limitation scales are only one of a number of factors relevant to
planning and management of services and personnel.
-- Note: Based on my knowledge of functional limitations scales, and the state-of-
the-art as far as program evaluation is concerned, it is difficult to answer
these questions as framed. An "ideal" system would be useful in areas where
current systems clearly are not I am assuming you mean an ideal system,
and will respond as if that is your intent. If not, please deduct 1 or 2
points from any of my votes relevant to usefulness of scales. Also, in future
questionnaires, please be specific as to what we are assessing.
-- The utility of a scale for these purposes, as for many others, depends on whether
it can be made relatively short and easy to use.
II-A-3-a
-- How would the scales be useful for this? Without some indication of the population
for whom measures are recorded, this is not answerable. If the population is VR
clients, certainly not. If the population were all respondents to the U.S. Census,
the answer is a conditional "yes" (depends on the reliability and validity of
the scale).
-- Am not sure of meaning of this item.
-- This item is not clear to me.
II-A-3-c
-- Major value lies here for state agency personnel.
II-A-3-d
-- Can be dangerous use if it locks program into status quo.
*
1 - vocational and medical consultation
*2 - standards for providers of services
*
3 - estimation of community resources needed
comments on opposite bage
11.
Importance/
Confidence
4. Federal Administrators
Please rate the importance of functional limitation
scales for the Federal administrator and your confidence
in your response
Please rate the importance of the following possible
purposes of functional limitation scales for Federal
administrators, and your confidence in your response
a) estimating the population at risk
b) program evaluation
c) set priorities in Federal program
d) develop programs to special target groups
e) provide basis for audit and program administrative
reviews
f) new agency reporting system
g) budget justification
h) establishing common program evaluation criteria
for all State agencies
i) initiate new legislation
j) other (list and rate)
Comments Concerning Page 10 (b)
II-A-3-e
-- It may be of some use in terms of documenting the number of severely disabled
served.
-- Here, as in b, C, d, you may be assuming systems which are not as yet in place.
Frankly, I do not see functional scales providing planning to management infor-
mation that would differ significantly from the current "% severe served" or
"% severe rehabilitated." Not that it wouldn't be more complex. But for many
of these program categories, it is not clear that providing more categories
for service quotas would serve the client (in terms of equity) or the program
(in terms of setting of priorities, etc.).
II-A-3-f
-- Only in that it would provide some quantifiable data.
II-A-3-h
-- Scale(s) would provide guidelines for resources required.
*
1 - vocational and medical consultation
*
2 - standards for providers of services
*
3 - estimation of community resources needed
Question
Importance
Number
4 3' 2 1
Comments Concerning Page 11
4
151
4 a
111
i
3
I
II-A-4
-b
2
4
I
-- Federal Administrators - Introduction of functional
measures will impact on priorities most directly in
- C
2
3
1
1
research and training because in those areas specific
- d
4
3
projects can be tied to specific assessment purposes.
- e
I
4
I
}
-- Functional assessment may provide a new way of defining
- 4
)
5
1
special target groups.
- 9
1
4
I
I
-- May provide a more straightforward way of reporting.
- h
33
1
-- Overall rating is lower here, because most of the
functions are inadequately defined in this question
-
6
(see comments), and with the exception of how RSA
might use the scales for reports to HEW and Congress,
-
the other functions are unclear and some are trouble-
-
I
some (e.g., "set priorities").
-
I
-- Functional Limitation scales are only one of a number of
factors relevant to planning and management of services
and personnel.
II-A-4-a
-- Again, not sure of meaning.
II-A-4-b
-- May be more relevant to program evaluation activity and development of theory
in the program evaluation office, than to actual measurement of program achieve-
ment.
II-A-4-c
-- May facilitate establishment of quotas, standards. But, the numbers themselves
will not set priorities. Nor would ! recommend a priority system based only on
functional limitation.
-- As in 3(d) there is a danger of fixing the program at status quo.
II-A-4-8
-- Some ambiguity here. If target "groups" are grouped by limitation rather than
by disability, it is important. If target "group" is identified by disability
-- of limited importance.
(Design team comment: I think the former meaning is implied).
II-A-4-e
I have a concern that here we are considering numbers and ratings for their
own sake, as components of check-lists for audits. Without knowing what one
might be reviewed or audited for, with respect to functional limitations scales,
I feel the question is difficult to answer. See my general note to Section II-3.
*
1 - identify research needs
*
2 - identify needs in training
*
3 - suggest guidelines for research support
comments
un
opposite
page
12.
5. Researchers
Importance/
Confidence
Con
Please rate the importance of functional limitation
scales for the researcher and your confidence in your
response
Please rate the importance of the following possible
purposes of functional limitation scales for researchers,
and your confidence in your reponse
a) development of new techniques for and methods of
coping with or overcoming handicaps
b) develop new technological devices or adaptation
of existing devices to specific individuals
c) developing aggregate files of special target groups
d) utilization in demonstration and pilot projects
e) measuring impact of alternative case delivery and
administrative approaches
f) other (list and rate)
In your judgement, for which group would functional
limitation scales be most important (client, counselor,
State administrator, Federal administrator, researcher)
Please write your response in the blank:
Please rate your confidence in your response
Comments Concerning Page 11 (b)
II-A-4-f
-- Again, it might be useful for reporting. But, without knowing what is being
reported to whom, for what reason, all I can say is that more measures will
generate more tables.
II-A-4-g,h
-- Would fulfill same programmatic function as the program standards do, and allow
report to Congress on service to severely disabled in "scientific terms".
II-A-4-i
-- Maybe.
-- Use may pinpoint deficiencies in service resources, modifications in eligibility
criteria and possibly suggest a basis for reorganization of federal human service
agencies.
*
1 - identify research needs
*
2 - identify needs in training
*
3 - suggest guidelines for research support
Question
Importance
Number
4
3'
2
1
Comments Concerning Page
12
5
6
5-a
I 3 3
II-A-5
-b
2
2
2
I
- C
6
I
-- I think probably the most compelling need for work on the
development of functional limitation scales probably lies
- d
6
1
in the university research departments, specifically the
5
existing network of rehabilitation research and training
- e
)
centers. I understand that some effort is being made at
4
I
this by the development of rehabilitation indicators pro-
jects at some units, research projects for weighted case
-f*2
1
closures at other units and probably many more in a variety
of different areas. It is my personal impression that a
-f*3
I
tremendous amount of research work is going to have to be
done in this area before we can come out with a viable
system of measuring functional limitations of clients.
To me the most logical place to begin this is in the research centers, although I
do not think that it should be specifically in medical centers that this process
should begin. However, the medical aspect of functional limitations is extremely
important but it is not the total picture.
-- The scales are important for researchers. But, first, research is important for
the scales. My votes re importance in this section are based on the fact that a
great deal of research remains to be done. a, b, and e are uses of scales that
might best be deferred until we know more about their reliability and validity
and the applicability of particular measures to particular design problems.
II-A-5-a,b
-- My low ratings of importance may be misleading. These are extremely important
tasks, but I question the utility of functional limitations assessment for develop-
ing new techniques and/or devices. The level of specificity at which we have been
working at least, does not seem sufficiently precise for that purpose.
-- 5 a) and 5 b) depend somewhat on the form and specificity of the scale. It would
have to show the extent of specific functional limitations.
-- Direct use is viewed as limited. However, identification of needs in substantial
numbers may stimulate treatment innovation (indirect use.) Another indirect use
is in provision of criterion measure(s).
II-A-5-c
-- Meaning unclear.
II-A-5-e
-- I would see more applicability in terms of defining client groups than for
measuring outcomes.
II-A-5-f
-- In many instances when randomization is not possible in studies, say, of effect-
iveness of a treatment measured by such variables as change in income, client
satisfaction, etc., a measure of functional limitation(s) may be more useful than
type of disability or other demographic variables as a control.
Comments on Most Important Use-Group
-- This item should distinguish between service delivery and analytical staff,
i.e., drop researcher category.
-- My response here, "client," is based on the fact that it is the client who might
have the "yes/no" decision of service based on the scale.
*
1 - reducing error rate in eligibility
*
2 - use as a control variable
*
3 - developing new definitions of handicap
comments on opposite pace
13.
6. Relevant Others
Importance/
Confidence
Please rate the importance of functional limitation
scales for the following relevant others in the
rehabilitation process and your confidence in your
response
a) trainers of rehabilitation personnel -- to focus
trainees' attention on client limitations rather
than their disability category
b) family members -- to measure progress, to increase
supportiveness and to improve the understanding of
the disabled person's problems
c) secondary service providers -- to focus their
attention on the needs of the client
d) other (list and rate)
Question ImporTance
Number
4
3
2
1
Comments Concerning Page
13
5
421
-0 -
133
II-A-6
-C -
2 1 3
-- Again, it depends on how much information the scale gives.
1
If there are no subscales it will not tell much about the
needs of the client.
of
2
I
0
-J*3
-- Relevant others - (a) I believe in the importance of FL
1
scales for training, but I object to "focus attention" as
vague. Prefer "professional skill acquisition".
II-A-6-a
-- Still some disadvantage in focusing on negative attributes.
II-A-6-b
-- This can be done without elaborate scales.
-- Again, assumes adequate validation. All of the earlier statements about clinical
application apply here.
II-A-6-c
-- This may have possibilities, but find it difficult to see how what results would
be communicated to whom in order for any impact to take place.
-- "Secondary service providers" needs more specification. I consider this to in-
clude rehabilitation facilities, training facilities, medical suppliers, etc.,
but others would view these providers as primary.
II-A-6-d
-- Representatives of the press, radio, TV, etc., can be taught to distinguish
between "dependent" and "independent" disabled people through concepts of
functional limitation. They, in turn, reach many other people.
(Design team comment: There are an infinite number of uses for functional
limitation instruments. We are interested in direct users in this section.
We feel if we asked questions concerning all possible uses, we would divert
attention from our main purpose -- that of describing the primary user groups
and their needs to instrument developers so that instruments they develop are
practical and useful to the primary user groups).
II-A-6-e
-- Unless doctors/nurses are "secondary service providers", they should be targets
for training/orientation to FL scales.
(Design team comment: See question 3-h, on page 10).
*
1 - media representatives
*
2 - general educators
*
3 - doctors and nurses in non-rehabilitation specialties
(Please write comments on opposite page
)
14.
B. Unique Client Populations
The purpose of this Section is to identify client populations presenting
unique problems in the development of functional limitation scales. In
this Section, please consider the consequences of the particular limitation
of each population on other life activities. It is these limitations which
may not be captured in a scale, which nevertheless, impact on the assessment
of functional limitations. For example, a person with epilepsy has no visible
handicap, nor is his handicap manifest in his overt behavior. However, this
person may not be able to work in a given environment because it may facili-
tate the onset of a seizure.
For each population, please rate the following: 1) perceived adequacy of
the research, encompassing all aspects of client behavior, identifying their
functional limitations (including those not usually thought of as being a
direct consequence of the disability); 2) the importance of developing ade-
quate measures for the particular population*; and 3) the feasibility
(practicality) of exploring these areas in relation to functional limitation
scales. Also, please rate your confidence in each response.
To stimulate your thinking we have identified one or more issues involved
for each client population. Please consider other possible issues when
giving your response.
Adequacy/
Confidence
Importance/
Confidence
Feasibility/
Confidence
1. Mentally Retarded
Physical capacity to perform life functions are present
but limited mental capacity may prevent realization of
optimal life functioning
2. Blind
Visual impairments can impede the realization of physical,
mental and social capabilities
3. Deaf
Hearing impairments can impede the realization of
cognitive, social and communicative capabilities
*Whether separate scales for various client populations
or incorporation of measures into a general scale should
be developed is dealt with at the end of this Section
Question
Adequacy
Importance
Feasion.Ty
Number
4
3
2
1
4321
4 3 2 1
Comments Concerning Page 14
1
16
52
2411
2
43
511
3 4
II-8
3
15
1
6
34
-- Believe there is a difference
among physical, mental and social
capabilities. Should be separated
out. Adequacy of research is greater in area of physical limitations, for example.
General: It is not clear what, in total, you are after here. Adequacy of research
relevant to functional limitations (e.g., measurement, causes, correlates, modifiers)
in my opinion is at best marginally adequate although better in some disabilities,
e.g., mental retardation.
Importance is uniformly high, regardless of disability.
Feasibility of developing functional limitation measures is high, regardless of
adequacy of research. Feasibility of measurement depends on technology of measure-
ment, not the adequacy of research with any given disability.
My bias is, that given consensus on the relevant dimensions of functional limitations
measurement can proceed in a logical and systematic fashion.
Feasibility should not be confused with validity and/or reliability which will vary
with the method of observation. In the case of non-visible disabilities, and in
the case of some limitations, e.g., attitudinal, we may need to rely on client self-
reports which may lower validity and/or reliability. However, feasibility is not
reduced.
You are inviting us to pick out "issues" of relevance to specific disabilities. By
this you seem to mean the domain of limitations which are of particular relevance.
I see this as relevant only if limitation scales are made disability specific, a
contingency viewed as remote.
Technology of measurement, however, may vary with disability, e.g., self-report
may need to be discounted with severely retarded, paper and pencil tests avoided
with blind. (Obvious examples).
-- NE for adequacy of current research. I have marked them all inadequate because
I doubt that there is anything which is in a form that could be easily translated
into a scale, but I assume there have been plenty of observations made and written
about.
With regard to importance, I think it is necessary to reflect the symptoms or
behaviors which reflect the stated condition or impairment. However, the reflection
does not need to be exhaustive for the purposes of developing a scale which can be
used. for determination of extent of limitations and which will aid in planning,
evaluation, and management decisions. It is necessary to be more exhaustive if
the scale is to be used as a diagnostic and service provision tool something
I doubt would be feasible.
-- It is difficult to answer many questions about specific diagnostic groups when
a major goal is to cut across these groups.
-- Note: I am having serious problem with feasibility rating throughout. The
no R & D required for a 4 rating is inappropriate. Research is going to be
required at a massive level throughout. Your definition of feasibility should
be reviewed. It should mean "It can be done".
(Design team comment: It has been rewritten).
II-B-1
-- Problem is acute in assessment of multiply handicapped MR.
II-B-2
-- Again, problem is that multiply handicapped deaf people are too often inade-
quately evaluated re: functional limitations unrelated in etiology, but difficult
to identify or measure, due to deafness. Examples: mental retardation, emotional
problems, visual impairment. Also, a hearing world imposes functional limitations
measures based on speech and written forms of communication, not communication
per se in the mode a deaf person may use. Thus, coding appears as in VR - "Deaf,
unable to talk"; "Deaf, able to talk". But speech does not take into account use
of other modes of communication, e.g., sign language.
(rlease write comments on opposite page
15.
Adequacy/
Confidence
Importance/
Confidence
Feasibility/
Confidence
4. Deaf-Blind
The combination of these disabilities may create unique
limitations in fulfilling cognitive, social and
communicative capabilities
5. Hidden Handicaps and Progressive Diseases
Below are listed conditions which are not directly
observable or manifest in behavior, but may limit
the type or extent of activity engaged in and/or
conditions which may cause increasing limitations
in activities over time
a) epilepsy
b) diabetes
c) renal disease
d) cancer
e) coronary artery disease
f) pulmonary disease
g) endocrine disorders
6. Mental Illness
Emotional handicaps can impede the realization of physical,
social and communicative capabilities
7. Recently Traumatically Injured
Traumatic injury usually brings about stress, depression
and similar complications which may or may not be picked
up on typical functional limitation items
Question
Adeavacy
Importance
Feasibility
Number
4 3 2 1
4321
4 3 2 1
Comments Concerning Page 15
1'4'2
--
4
-
2321
5-a
16
51
213
II-B-4
-b
24
I
61
16
-- Etiology and mental orientation are
-C
24
1.
52
151
key factors. Client history must
identify if deaf-blind person is a
-d
25
61
151
pre-lingually deaf person with an
acquired visual limitation, a post-
-e
25
511
142
lingually deaf person with visual,
-f
16
51 I
I 42
etc., or, are we working with a con-
genital blind person losing hearing?
-9
142
61
223
Age at onset and etiology variables
6
25
61
2.221
for both blind and deaf persons are
major areas needing improved measures.
J
34
52
114-11
II-B-5
-- Endocrine disorders. Metabolic
disorders, particularly associated with mental disturbance, have received little
research attention in assessment of functional limitations. Hypoglycemia has
been shown to be responsive to treatment by diet, but again functional measures
are few. Thyroid disease is another area, primarily pathological, but where
functional measures could assist persons requiring help in addition to that of
the physician.
-- Progressive diseases present special and very difficult problems, particularly if
a goal is prediction of vocational outcome. Even if the description is accurate
for the present, I am not sure how to take into account the likelihood of future
change.
-- I think this is probably one of the areas in which most research'and most attention
should be given to the development of functional limitation scales that is in the
area of mental functioning and also in relation to the other hidden disabilities
that are exhibited in the questionnaire. In this connection, I think a team
approach to the measurement of client limitations is essential before any realistic
system of measuring impairment can be established. While the medical in the case
of hidden disabilities is very important, the implications for the psycho-social,
the economic and the personal functioning of the individual come largely into
question here. It is my observation that people with hidden disabilities usually
have an emotional reaction to the disability itself revolving around the fact that
they appear healthy and are frustrated in the attitudes of others around them as
to why they are not functioning as a so-called "normal" human being. I have also
observed that in many cases the two extremes are exhibited by families of individuals
with these problems. One is rejection of the individual and the other is over-
protection. Either of these polarized points can create extreme difficulty for
individual clients and they probably require the efforts of a team in order to
arrive at some reasonable estimate or measure of the extent of the functional
limitations imposed by the secondary, that is emotional factor surrounding the
hidden disability.
-- These disorders are too varied to make judgements with confidence.
II-B-6
-- We have some difficulty with this population using our scale, but we are unsure
about the cause. It may be that it simply takes longer for relevant vocational
data to be covered in the interviews with these clients so that ratings based on
initial interviews (as we have been doing) are incomplete. Another possibility
is that our psychological and social items are not detailed enough.
My adequacy ratings consistently reflect my impression that functional limitations
measurement with all diagnostic groups has a way to go before clinical application
is feasible.
(Please write comments on opposite page
)
16.
Adequacy/
Confidence
Impor tance/ Importance/
Confidence
Feasibility/
Confidence
8. Other Populations (list population and issues and
rate)
9. After responding to these items, do you feel
researchers should attempt to develop a) scale irregard-
less of disability type or b) more than one scale based
on disability type? Please keep in mind 1) the desire
for a single universal scale, 2) user groups and their
need, 3) possible problems associated with a client who
has multiple disabilities, and 4) unique problems
associated with a given type of disability
Response: Should try to develop 1 scale
Should try to develop more than 1 scale
based on disability type
Don't know, unable to determine with
present state-of-the-art information
(Please indicate response with check mark () in first
box and rate confidence in second box)
Question
Adeavacy
Importance
Feasibility
Number
4
3
2
1
4
3
2
1
4
3
2
1
Comments Concerning Page 16
8#
=
to
1
II-B-8
8*³
3
-- While the current trend in rehabili-
9
tation has turned away from the
Should try to develop 1 scale
3
socially disadvantaged, this group
Should try to develop more than
can be encompassed reaily under con-
1 scale based on disability
cept of functional limitation(s).
type
2
-- Both MS and MD are examples of the
Don't know; unable to determine
need for improved ways of measuring
with present state-of-the-art
limitations when the prognosis is
information
1
deterioration of function.
II-B-9
-- Uniformity is needed in lexicon,
concepts, and methodology of functional scales. Standardization of validity tests,
training of appropriate users, and criteria for incorporation into management infor-
mation systems would be desired. But one scale would be overbearing in detail or a
disservice to specific disabilities. Hence we need a systematic way of identifying,
evaluating, and "accrediting" approved scales. But one single scale - no! We are
not there yet!
-- While I favor measurement of functional limitations independently of disability, I
believe it may be necessary to measure at two levels:
Level 1. Screening to determine global levels for each specific domain.
Level 2. Detailed measures for each domain identified as important at Level 1.
-- Earlier in this questionnaire, it was implied that a functional limitation scale
could focus on need, not disability group (Section A-6). Would separate scales
serve this function?
My vote reads: "It.is preferable to develop one scale, but I am not sure how
feasible this is given the state of the art. However, one reliable, valid scale is
probably no more difficult than a number of distinct, specialized scales."
-- I would prefer to see a single scale for all physically disabled. It may be neces-
sary to have separate scales for MI and/or MR populations. Further, separate scales
will probably be necessary to meet the needs of various disciplines -- e.g., I am
fairly sure that a scale designed for use by VR people would not be satisfactory
to physicians or physical therapists.
-- I am still having some difficulty at this point with the concept of scale. I think
again an assumption is being made here that functional limitations can be measured
on one type of scale that can become a part of the case file and will serve a
variety of purposes. I think this is an erroneous assumption. The field is not
nearly far enough along in its research efforts to make an estimate or an assumption
at this time that one scale can be developed which will handle all the functional
limitations. In all probability the most that can be anticipated from this would
be a profile of functional limitations. This profile would be based upon a variety
of sub-measures and scales which would indicate the wide range of functional limi-
tations imposed by an individual's disabling condition. Also whatever scale is
used must provide for documentation and verification of the findings of the individual
submitting the information on functional limitations. This will vary from client to
client in relation to whether we are dealing with a physical or a mental problem.
Either of these problems would require different forms of measurement. I think the
basic dichotomous approach to measurement in this field should center around physical
disabilities and mental disabilities and the mental clouding that may result from
either a physical or mental disability. Current rehabilitation legislation deals
with two disabling entities. One is physical, the other is mental. I think also
in this respect attention should be given to measurement of the functional limi-
tations imposed by the so-called behavioral disorders such as alcoholism and drug
addiction. The consequences of either of these diseases are extremely serious and
all of us who have worked in the field of vocational rehabilitation are acutely
aware of the problems imposed by them. I stand by my marked response on page 16
that we simply don't know enough at this time to make any reasonable estimate of
what is needed in the final format of a scale for measuring functional limitations.
*1 - learning disabled
* 2 - socially disadvantaged
*3 - chronic pain, especially low back
(Prease write comments on opposite page
17.
C. Criteria for Functional Limitations Scales
To be useful, functional limitation scales must meet a variety of criteria.
Some criteria are more essential than others, and at times, some criteria
are not practical when measuring human qualities. Please rate the impor-
tance of and feasibility for meeting each of the following criteria.
Importance/
Confidence
Feasibility/
Confidence
1. Measurement-Scoring Features
a) nominal, ordinal, equal interval scoring
Scoring of items on scales may be nominal
(descriptive only, cannot be ordered, or
added), ordinal (scaled items in ordered
sequence -- greater or lesser -- but items
not at equal intervals), or equal interval
(scaled items in order at numerically equal
distances on the property being measured.)
Please rate each of these criteria:
Nominal
Ordinal
Equal Interval
b) overall score of severity
c) use of subscales
Different user groups may need different levels of
specificity on information regarding functional
limitations. For example, the counselor may need
highly detailed information on client limitations,
while a researcher may need only a general indi-
caton of a class of behaviors (e.g., general
mobility abilities) while the State Administrator
might need only an overall score of severity. On
the other hand, some rehabilitation professionals
feel such scales should be brief, which may inhibit
the development of scales which are based on sub-
scales
d) weighting items in relation to one another (keep
in mind the necessity of doing this for determining
a score of severity, and the decisions involved in
equating different types of limitations, e.g., how
do you weight physical limitations in relation to
mental limitations)?
Question Importancelf easibility
Number 4 3 2 1 4 3 2 1
Comments Concerning Page 17
÷
-noromal
1
1
4
I
412
II-C
Ordmal
34
16
-- I think this entire section of measurement
equal
interest
142
1231
and scoring features in reliability and
validity, brevity, detail of scale, etc.,
-b - b
331
133.
are far too early in the research procedure
-C - C
521
341
to make any determination at this time as to
what their ultimate form should be. I think
- d
242
44
it matters little at this time which it should
be nominal, ordinal, or equal interval measure-
ment approaches. I think we should first look
at a thorough analysis and definition of functional limitations and then make some
serious probes as to what is necessary to be able to measure them and then apply
whatever statistical procedures are necessary. The ultimate result of this could
be any or all of these in any variety or number of combinations.
II-C-1-a
-- Nominal measures are viewed as of limited value. Ordinal measures are essential.
Equal intervals are illusory, but do permit greater flexibility in statistical
treatment.
-- An ordinal scale is essential and an equal interval is highly desirable, but I
doubt if the latter is feasible.
II-C-1-b
-- Overall value of severity has uses in program evaluation, counselor performance
evaluations, etc.
II-C-1-c
-- Use of subscales, at least subscales for each domain are mandatory.
-- Perhaps what we might get with subscales is a profile (e.g., somewhat
resembling an MMPI profile) which will show areas of relative strength
and limitation.
II-C-1-d
-- Here, again, we are talking about programmatic concerns in a void. Weighting
has been a concern in VR for years. Obviously it is "important." But without
knowing how such a scale would be used (counselor promotions? client eligibility?
sanctions related to state performance?) it is difficult to say how "important"
or how feasible.
-- It may be that if you develop subscales you may not need to weight items. Any
weighting would need to be based on empirical grounds. Chances are great that
weights developed on one population would not hold up to cross-validation.
-- Item weighting in measurement has had mixed reviews. Should be attempted as
one method of striving for greater internal consistency.
(Please write comments on opposite page
18.
Importance/
Confidence
Feasibility/
Confidence
2. Reliability (consistency of the scale, obtaining the
same results)
a) test - retest reliability on a given client
b) agreement among 2 or more raters
3. Validity (does the item/scale really measure what
it is supposed to measure?)
On scales such as this, true validity measures do not
exist, and one must seek the best available substitute.
Below are some of these substitutes. Please consider
each carefully and rate them
a) level of agreement with counselor assessment
b) level of agreement with other functional
limitation scales
c) level of agreement with client assessment
d) prediction of behavior in a wider range of
settings
e) other
4. Brevity/Detail of Scale
These criteria are presented togehter as they bear a direct
relationship to one another. Brevity is often mentioned
as a desirable trait for functional limitation scales as
the user groups may be extremely busy and have little time
to spare. On the other hand, some professionals feel that
detail on such a scale would provide the counselor and
client with important, specific information helpful in
setting up the clients' rehabilitation plan. Also the
more detailed the scale, the more it forces the user of
the scale to look at all of the specific needs of the
client. Please rate the following:
a) brevity
Question
Number
4
3
2
1
4
3
2
1
Comments Concerning Page 12
2- q
43
16
-b - b
5
2
24 I
II-C-2
3-a
15
-
24
- b
132
14
I
-- One should be able to expect a reasonable
degree of consistency over time, but some
-C
122
14
-
change should be expected of the individ-
- d
312
133
ual's situation or condition changes.
Inter-rater reliability is the more impor-
- ex
1
1
tant measure.
-e*2
-
I
1
-- Test-retest reliability is important to
4-a
222
32
1
determine to assure that limitation is of
substantial duration but has reduced
-b - b
33
42
feasibility for some measures e.g. attitu-
dinal, on some methods of observation e.g.,
self-support.
( Design team Comment: We are referring here to the issue of reliability in
scale development methodology).
II-C-3
-
-- Validity does subsume concept of content validity - partially covered by 7
(Descriptiveness).
Validity should not be based on a single external criterion, but as many as
is economically feasible. Use of standardized tests of capacity/achievement/
intelligence should be reviewed thoroughly.
-- On validity: Counselor and client assessments are important -- the main
constraint is the extent to which the purposes and limitations of the
measures are transmitted to the clients. Comparison with other measures is
a peculiar suggestion, given that there are a number of inadequate measures
and measures of alternative functions. Which measure or measures would one
choose? While this makes sense theoretically, I don't know that I would
recommend any other measure for a validity check.
Validity issue depends on what the scale is "supposed to measure." Some
scales only offer to assess what the person "can do," others what the person
"typically does," others what the person's "potential" functioning is. To
rate validity issues, one needs to know the data-gathering paradigm and the
types of statements that will be formulated, based on the measures.
-- Are you certain that "true" or more traditional validity measures do not exist?
What about work samples and other direct measures of functional performance?
(Design team comment: Please refer to item 3-b)
II-C-3-a
-
-- The functional limitations inventory will be a type of counselor assessment,
only more organized than the usual "clinical impressions"
II-C-3-b
-
-- Assuming that there are other FL scales intended for use by counselors.
II-C-4
-
-- With increasing detail the rater can run into increasing problems because of
gaps in knowledge about the client.
*1 - careful research/demonstrations designed to
test alternative measures
*2 - standardized measures (e.g. Vineland Social
Maturity Scale)
comments on opposite page
19.
Impor tance/
Confidence
Feasibility/
Confidence
5. Objectivity-Subjectivity of Items
Some rehabilitation professionals feel that objectivity
is necessary in such a scale as it reduces reliance on
judgement, while others feel that the judgement factor is
important. An objective scale would more likely be
reliable, but would necessitate the construction of a
longer scale. A subjective scale might be shorter, but
would be more difficult to work with in constructing a
scoring system. Please rate the following:
a) objectivity of items
b) subjectivity of items
6. Predictiveness
Prediction would provide the users with information about
a client's rehabilitation potential. It would refine the
eligibility criteria, thereby insuring services are de-
livered to the target population. The disadvantage is that
particular strengths/weaknesses of a given individual may
not be sufficiently accounted for. The measurement
techniques may require equal interval scales, or complex
weighting.
7. Descriptiveness
A descriptive scale would provide information for coun-
selor in developing client plans and in assessing client
progress. The measurement techniques required for
description would not be complex, but a problem may
be the creation of an elaborate taxonomy with reliance
on semantics which have no further utility.
8. Prescriptiveness
This criteria has the advantage of providing the users
with information about rehabilitation services which need
to be provided (prescription of services). This may rein-
force decisions made by rehabilitation professionals, but
may cause resentment because of "dehumanization" of the
decision-making process. It may also require the use of a
lengthy scale because of the possible number of prescrip-
Comments Conerning Page 18 (b)
-- I have marked 4 for brevity and 3 for detail. As you have noted, it depends on
the expected use. For counselor use, the detail may be necessary, but for
evaluation use it is usually not. I would urge that a feasible evaluation
instrument be developed whether or not it fits counselors'. needs for case
planning.
-- Using a branched or gated structure, one can obtain detail at a desired level
and skip areas that are non-problematic for a client (for the sake of brevity
of data gathering).
-- Brevity or the lack thereof should be empirically determined i.e. does a short-
form scale demonstrate sufficient reliability and validity.
II-C-4-b
-
-- Some of the descriptors under Level 4 Feasibility do not fit my rating here.
What I mean is that if you set about to do it, writing a long, detailed scale
is always possible you just keep adding items. The tough thing is to make
it compact.
Question
Importance
Feasibility
Number
4
3
2
1
4
3
21
Comments Concerning Page 19
5-a
62
341
b
214
313
II-C-5,6,7
6
512
152
-- As far as objectivity, predictiveness,
descriptiveness, etc., are concerned,
7
313
33
1
I think these two must be a result of
the basic research that is done in this
8
2311
1411
field. We first have to determine what
is possible then determine the uses that
can be made of the data and the techniques gathered. I think in the final analsis
the scales that are used or whatever measurement system should be as objective'as
possible. I think they should be consistent from one situation to another and I
think there should be some kind of a national network so that all of us who are
involved in the process of measuring functional limitations would have some common
basis of understanding. From the standpoint of practical and vocational rehab-
ilitation, I think the prescriptive potential of a measurement system such as this
has considerable merit. In effect that's what we are doing now, we are buying
medical and psychological information which prescribes certain things that need to
be done and on the basis of that imformation we are predicting what might happen to
the client. Certainly the information that we gather about our clients and the
reports we purchase is of a descriptive nature. All of these are essential and it
stands to reason that you cannot have a system without all of these elements being
present.
II-C-5
-
-- Our scale is a mix of objectivity and subjectivity. It seems preferable to make
the scale as objective as possible, given the conflicting needs to keep it rela-
tively brief and usable with a limited data base, If the scale were lengthy and
entirely behavioral, however, a counselor would probably be unable to complete it
solely on the basis of interview data. He/she would need to observe the client
over time in a variety of settings. Allowing for some inferences and judgement
reduces the time required and the number of items required,
-- Extremely difficult to judge in the terms given, and depends on the domain of
limitation and method of observation.
-- The fact is that, whether the scale is constructed in an objective or subjective
fashion, it will be used as objective information. The validity and reliability
problems of subjective judgments aside, the choice here seems to clearly be
"objective." Or call the scale "opinion as to functional limitation."
-- Objectivity is highly desirable but there are many "shortcuts" allowed by
subjective ratings which are probably necessary to get a feasible instrument.
I believe adequate reliability can be obtained on many somewhat subjective items.
II-C-b
-
-- Subjective items are potentially extremely harmful. "Importance" scale is
irrelevant, would rate it 1/2 on "desirability" scale.
II-C-6
1
-- Predictiveness should not be considered only for prediction of outcome, but also
for predictiveness of potential benefit of spedivid services, i.e. for predict-
iveness of successful process procedures.
-- The state-of-the-art would require much more research before one could begin to
lead with predictions (need a larger sample of clients, for whom scale measurements
have been recorded, and longitudinal information, before this will be possible.)
-- There are still many unanswered questions and much to be done. I am hopeful,
however, that it will be workable.
Please write comments on opposite page
20.
Importance/
Confidence
Feasibility/
Confidence
9. Ease of Interpretation
Users of functional limitation scales may not be well
versed in scale usage and may not be able to utilize a
more complex scale. Also, users are typically busy
persons who need easily interpretable information. On
the other hand, for ease of interpretation, you may sacri-
fice detail which might be needed to develop an effective
rehabilitation plan.
10. Ease of Training in Usage
Assume that implementation of a functional limitation
scale on a National level is a goal. State agencies have
limited resources for which to train personnel. If scales
were implemented, the easier it is to train personnel, the
more readily they would be accepted, and there would be
fewer problems in using the scales. Is this factor
important enough to be an essential criteria, or would
this be of secondary importance to other criteria such as
validity; reliability; meeting user group needs, etc.
Give. careful consideration to these questions when rating
the importance and feasibility of including ease of
training in usage as a criteria for a functional limitation
scale.
11. Other Components (list component, and issues, and
rate)
Comments Concerning Page 19 b
II-C-7
-
Not entirely sure of what you are seeking here. I say it is important to have
content validity necessary in order to describe limitations in as many different
dimensions as are necessary to provide differential service or to provide a useful
measure of outcome. For example, some attitudinal limitations may have no bearing
on amount or type of service required and a change as a function of treatment,
although socially significant, may be beyond the mission of vocational rehabil-
itation.
-- I have previously expressed my reservations about using a FL scale to measure
client progress.
-- Unclear description which reads as heavily biased, and opposed to "descriptiveness."
Whatever is meant here should be asked again in a more careful question.
II-C-8
-- I can't imagine a situation in which a FL inventory would go farther than
suggesting the possibility of certain services for a client. To demand that they
be given regardless of the judgment or wishes of counselor or client seems
ridiculous. Conversely, to withhold them because of a score in the face of
apparent need would be unwise unless strong supporting justification can be made.
-- I think it is possible to combine the qualities of descriptiveness and prescrip-
tions to get a feasible instrument.
-- Prescriptiveness would be enhanced if thought of in terms of outcomes to be
achieved instead of services to be provided.
-- Here, "criteria" is mentioned for the first time with "purpose" in mind,
Earlier questions assumed "prescriptiveness" as a possible function of the scales
(A-2, (b), (c).) Prescription cannot, in my opinion, be left to scales. I
would reinforce my earlier votes that this is not an important purpose (i.e.
criteria).
-- Less feasible than important!
Question
Importance
Fearbility
Number
4
3
2
1
4
3
2
1
Comments Concerning Page 20
9
4
3
251
10
251
3 5
II-C-9, 10
1
1
-- These criteria should not be given serious
consideration in the long run. Unfortu-
nately these are factors relevant to accept-
ance by agencies and counselors. However, I feel that the technical expertise
required is not excessive.
-- Ease of interpretation and usage are highly important: without them, I doubt that
a FL approach will ever be widely implemented. On the other hand, they have to be
secondary to the criteria of reliability and validity and the ability of the scale
to meet user group otherwise the whole effort will be useless. What good
is a quick and easy scale if it doesn't tell you anything?
II-C-9
-
Medical model of using consultants specialized in a field to help interpret should
be considered an alternative to making the indicators/scale itself easily interpreted
by all users.
-- As my most important user was "client," I would vote for clarity. However, my
second most important user was "researcher," and detail is important there. I
would prefer, as expressed in B-4, an expression that is brief and easy to comp-
rehend, plus addess to the assumptions and calculations that create the scale
(so that researchers and managers do have the detail, as needed).
-- No one scale will do everything. I would opt to get a usable evaluation instrument
first, rather than a diagnostic and prescriptive instrument (if there must be a
choice).
II-C-10
-- The training budget should not be the major design constraint. You will never have
validity, reliability, or uniform information without instructions on how to do the
measures. This question is, in fact, a complex question.
First: Is training essential?
Yes
Second: If states have limited training
budgets, should the scale be modi-
fied so that training would not
be expensive?
No
Then: If national scales are a goal,
should states be offered training by
RSA (short-term grants or ?)?
Yes
(Design team comment: Your point will be included in the final report)
-- Validity and reliability are essential. Then you can work on ease of training.
II-C-11
-- Economy of measurement is a practical issue. However, I am convinced that a
good system of measurement of functional limitations should be no more expensive
than the conceptually loose system now prevalent in most rehabilitation facility
evaluations.
* 1 Specialist staff to assist in use
(Design team comment: this is on utilization issue)
* 2 economy of measurement
comments on opposite page
21.
III. COMPONENTS OF FUNCTIONAL LIMITATIONS SCALE
The following list contains some of the categories of components for
functional limitations scales. Please rate the desirability and
feasibility of attaining information on these components in functional
limitations scales.
A. Strict Functional Limitations Items
The items listed below are true functional limitations information
(i.e., fit the definition of functional limitations.) Please rate
the desirability and feasibility of including information on these
variables in functional limitations scales.
Desirability/
Confidence
Feasibility/
Confidence
1. Health Limitations (medical problems; pain/
discomfort; etc.)
distances) 2. Mobility (Manual skills; locomotion - short/long
3. Communication (sending/receiving information to/
from other persons)
4. Cognitive-Intellectual (learning; manipulation of
symbols and objects; transference of learning to new
situations
5. Social-Attitudinal (acceptance of self; acceptance
of others; motivational)
6. Self-Care Skills (bathing, grooming, eating, etc.)
..
7. Others (list and rate)
Question
Feasibility
Number
4
3
2
1
4 3 2 1
Comments Concerning Page 21
1
4 3
I
= 3 1
2
71
3 4
III-A
3
8
2 5
-- All of these items are essential to an effective
4
6 2
3 4
measurement of cases that we are dealing with.
5
3 3
2
1 5 1
At this point in time it is extremely difficult
to say which of these is the most important, but
b
: 2
3 't
certainly attention must be given to all of these
areas before we can make any kind of affective
7ʳ
1
1
assessment of clients we are dealing with. This
it
1
1
will be even more true in view of the fact that
we are coming into an era of independent living
1
1
and comprehensive services to clients.
Te4
1
1
-- Note: Weights as to importance depend to a great
7.
1
I
degree on the use of the scale. I have given
highest weight to those areas where change might
T.
I
1
most be expected through rehabilitation services,
less on those areas where capability, not achieve-
I
1
ment, seems to be the issue.
-- Domains 1, 4 and 6 are highly important and can be quite feasibly measured. Each
domain should be separated into subscale. Manual skills should be removed from
mobility and subsumed under manipulative-aptitudinal. Another important category
often overiooked in many evaluation models is activity or energy level often modified
directly by the medical disorder, e.g., thyroidism or indirectly by psychological
factors. Good technology for measuring this is not available.
-- Re-read your definition.
II-A-1
-- Note: Only health is described in negative terms in this list. All other items
are expressed positively. Is there a positive health indicator that would be useful?
-- "Health" is not "functioning," nor "tasks," nor "roles" -- health problems may
underlie functional limitations, but are not themselves functional limitations.
II-A-2
-- "Mobility" seems to cover all physical motor skills -- not just an individual's
mobility (getting around) but reach, fine motor ability, etc. Are these areas
no so distinct that they warrant more than one category?
II-A-5
-- The social attitudinal items should be expressed as behaviors which demonstrate
or fail to demonstrate tolerance for working 8 hours a day or ability to get along
with others not attitudes.
-- "Self-acceptance," etc., is not "functional," nor "roles," nor "tasks" -- it is
a psychological construct.
*
1 - Emotional (situational appropriateness, balance)
*
2 - Manipulative, aptitudinal
*
3 - Productive-energy level
*
4 - Vocational Skills
*
5 - Social competence
*
6 - General Educational/Vocational Skills
*
7 - Homemaking/Child-care
(Please write comments on opposite page
)
22.
B. Other Direct Supportive Information
The use of functional limitations scales might necessitate the inclusion of
other direct supportive information if they are to be effective tools. Below
are listed several kinds of information which may be important and/or feasible
to be included into a functional limitations scale.
Importance/
Confidence
Feasibility/
Confidence
1. Exceptional Positive Personal Characteristics (e.g.,
unusually attractive appearance to employer, extremely
bright, extreme family supportiveness, wealth)
2. Exceptional Negative Personal Characteristics (e.g.
extreme ugliness, extreme unsociability, etc.)
3. Vocational History
4. Demographic Information (age, sex, etc.)
5. Environment
Functioning is somewhat dependent upon environment, and
one might argue that a functional limitation scale should
measure client limitations within a given environment.
Also, realistically it must be remembered that a given
client functions in a variety of environments. There are
various issues involved in dealing with this topic. Please
rate the importance and feasibility of the following items
and options, after giving some thought to the potential
problems, and benefits of each
a) Exclude all environmental questions; they should be
picked up through the measurement of other variables
(if you agree with this statement check the follow-
ing box and move on to section IV).
b) Please rate the importance and feasibility of in-
cluding the following types of environmental factors
on functional limitations scales:
i) attitudes of others
ii) presence of technology relevant to the handi-
capped person's needs
iii) availability of services
iv) physical accessibility of home, work and
community
Question
Importance
Fensibility
Number
4
3
2
1
4
3
2
1
Comments Concerning Page 22
|
21
4
3.1.12
2
11
5
3112
III-B
3
2212
51 1
4
-- I'm not really sure what is being asked for here.
21.22
6
I
I am assuming that the exceptional positive and
5-a
# who checked
box
=
3
negative characteristics measured might be related
5-b-i
to critical incident type measurement. It appears
12
1
131
that this might be some kind of a polarized
-ii
I
}
I
I
3 I 1
approach and only the outstanding characteristics
-11111/1211
of clients should come into play. I also have
iv
12
1
122
some difficulty as to what is meant about the
attitude of others in relation to clients. Cer-
21
1
212
tainly attitudes of others are important, but I'm
not sure how important the attitudes of others
are in relation to the highly technical data that
we need in our case files in order to substantiate
the findings of eligibility. I suppose my difficulty with this item revolves around
the client's interpretation and the attitudes of others. If that is the case, I
think then that is a counseling problem that should probably be handled on a one-to-
one basis with the client and the counselor. If it refers to parental attitudes, then
I think it is again something that should be handled in a counseling situation.
-- The items shown in III-B should be measured, but they should not be shown as functional
limitations. Functional limitations measurement should tell what people can do. It
should be valuable to identify homogenous target groups. The other supportive infor-
mation may be necessary for prediction of outcomes, but it is important that they be
measured separately from what the individual is capable of doing functionally. Inci-
dentally, I think it is necessary to have separate predictive scales for different
objectives employment, homemaker, or independent living -- but such scales would
include items in addition to functional limitations.
-- Overall, these issues reflect conditions which may constrain client achievement, or
may influence success. However, B-1 and B-2 seem totally inappropriate. B-3 and
B-4 may be relevant to employers, but certainly are included as usual employment
information and need not be reduced to a scale (if they are, perhaps the scale could
be modeled after bank charge card credit rating scales or such scales in toto be
included as some kind of "reliability indicator").
-- These variables and the environment are not functional limitations. Their inclusion
on a functional limitation scale simply leads to (and seems to stem from) conceptual
confusion.
-- Since no single measure or set of measures can be used solely by themselves, it is
reasonable to consider functional limitations along with all other factors which may
relate to process and/or outcome. Obviously, not all extraneous information is of
value but the type of data cited seems reasonable.
The "how" of using this information now comes under "clinical judgement". However,
if extraneous data is collected uniformly under standardized instructions on a
large scale, it can be integrated into a statistical prediction model.
-- Data (concerning B-1-B-4) is or should be recorded as part of the client evaluation
process with or without the use of functional limitation scales.
III-B-2
-- If the basic scales concern functional limitations, these traits would simply repre-
sent the extreme score on the relevant scale. I doubt that they would need to be
separate items.
III-8-4
-- I would prefer to see the functional implications of these traits dealt with in more
general items - e.g., age in terms of strength and employer prejudice. I am aware
however, of the many studies which identify these demographic variables as predictors
of vociational success.
23.
Impor Importance/
Confidence
Feasibility/
Confidence
c) Please rate the importance and feasibility of
including items which measure the client's ability
to utilize or cope with environmental factors to
his advantage:
i) attitudes of others
ii) presence of technology relevant to the handi-
capped person's needs
iii) availability of service
iv) physical accessibility of home, work and
community
v) transportation
Comments Concerning Page 22(b)
III-B-5
-- The "environmental barriers", b-(i) - (v) are definitely also possible con-
straints on achievement, and important in a comprehensive measure of difficulty.
But the complexity of such estimates renders them virtually infeasible, particu-
larly all respects to available technology.
-- Note that 5. (a) was checked. All items in 5. (b) could be relevant extraneous
information, but see no reason for "including
factors on functional limi-
tations scales.
-- We have been assuming full use of available technology in doing our ratings --
i.e., the ratings reflect residual limitations when all assistance is utilized.
When dealing with a client who does not have access to appropriate existing
technology or who has problems which technology is likely to handle in the future,
we seem to have the reverse image of the client with a progressive disability.
The present situation is likely to change, but this time in a positive direction.
-- Functional limitation scales need to specify the environment that is applicable to
test conditions (i.e., natural environment, rehab facility, etc.), but the environ-
ment is not a "functional limitation" per se. One must go with the definition of
functional limitation that has been provided or scrap it, but this must be consis-
tently applied throughout this document. The environment is not included in your
definition.
Question Importance
Probubility
Number
4
3
2
1
4
3
2
1
Comments Concerning Page 23
6-a
5 I 2
I
5
I
-b
-
42
I
I
4
I
1
)
IV-C-6
-
I
/
1
-- As in any other measurement device in the psycho-
Desirability
feasibility
social-medical fields, there is a major danger in
D-
431
161
premature application. Again -- it is my firm
-2 2
53
35
belief that satisfactory measures (not just on
unidimensional scale, a naive and simplistic
-3
421
142
idea) will require enormous expenditures in
-
4*3
1
)
developmental monies. Consequently there will
be political pressure to use the information
too soon.
Also, related to a prior point, without good developmental research and concomitant
professional training, misinterpretation will be a danger.
IV-C-6-a
-- Probably more likely than "overutilization" is the possibility of "underutilization"
of scales not being applied when they could be helfpul.
IV-C-7
-- Potential use in theoretical research, with potential humanitarian value, should
accrue from development of scale.
IV-D-3
-- Too vague.
-- Final point: Had serious difficulty in using your guidelines for the feasibility
scales. Other ratings were clear to me.
Question importance
Feasibility Y
Number 4 3 2 1 4 3 2 1
Comments Concerning Page 23
C-i
31
I
I
1
2
-
31
12 1
-iii
III-B-5-c
31
13
- IV
31
13
-- There seems to be a very subtle distinction
- V
31
between this Section and the preceding one.
13
I doubt that it is useful to distinauish
between them. We have been trying to measure
the functional impairment the individual ex-
periences regardless of etiology -- e.g., if a person has no transportation, the effect
is the same regardless of whether the problem stems from lack of a transportation system
in the community or the person's inability to use one that does exist.
In a different sphere, the inability to life more than 1- lb. weights on the job may
stem from such diverse causes as:
1 - Cardiovascular insufficiency
2 - Muscular weakness
3 - - Paralysis
4 - Back pain
5 - Personality disorder, etc.
We feel that it is preferable to record the inability to lift weight rather than to
measure all of the possible causative factors.
(Please write comments on opposite page
)
24.
IV. UTILIZATION
Adequate plans must be prepared to ensure that research findings and scales
(after development and field testing,) will be properly, effectively and
efficiently used. This Section deals with research utilization issues.
A. Facilitation of Utilization
Please rate the desirability (probable effectiveness), the feasibility
(practicality), and importance of each of the following methods of
facilitating the utilization of research. Also, please rate your confi-
dence in your responses.
Probable
Effectiveness*/
Confidence
Feasibility/
Confidence
Importance/
Confidence
1. Incorporating the use of functional limitation scales
into the training programs of rehabilitation professionals
(including in-service training)
2. Design of technical assistance teams to assist State
VR agencies in implementing the functional limitation
concept
3. Design of technical assistance teams to assist
facilities and other programs in using functional
limitation scales
4. Design of seminars/conferences to relay findings
to researchers, administrators and potential users
5. Design of seminars/conferences to introduce clients
to functional limitations conceps and scales
6. Funding of demonstration projects
7. Encouraging advocacy through National organizations
for the handicapped
8. Lobbying for and supporting legislation focusing
efforts on the conceptualization of disability based on
functional limitations
9. Others (list and rate)
*Use the definition of "desirability" in the front of
this Questionnaire for the definition of probable
effectiveness
Prsbable
Question
effectiveness
Feasibility
Importance
Number
4
3
2
1
4 3. 2
1
4 3 2 1
Comments Concerning Page 24
I
53
17
513
2
53
431
3:5
IV-A
3
53
431
26
-- Effective utilization will depend
4
521
62
341
upon proper sequencing of events.
5
3212
332
2213
Firstly, there will need to be
several years (5-10) OT intensive
6
521
71
52.1
measurement research, this phase
7
7
1
62
521
costing several million dollars.
Unless this level of commitment is
8
34
I
143
4311
forthcoming utilization is futile
and extremely naive.
9*
1
1
1
Following research it is essential
that highly visible demonstration
projects be used. This taking another 3 years. Concurrent with this training pro-
gram can be initiated a technical assistance program set up on a pilot basis with
both training and technical assistance being dovetailed with the demonstration pro-
jects.
Legislation is important, but legislative commitment must come substantially before
the utilization phase, special legislation will be required to provide the initial
research.
Client seminars are seem as of marginal value but advocacy organizations should be
involved from the start to assure full cooperation during the research phase.
The most critical think is for all to realize that what is being proposed is a
radical and comprehensive revision of the rehabilitation program.
-- All votes assume that an "ideal" system has been developed.
-- Utilization: There is a pressing need for order and authority, promoting
standardization in use of functional limitations. The rehabilitation field
needs some form of National Functional Limitations Standards Institute or
similar structure, composed of leaders from research, services, consumers,
etc., with the prestige to lend authority to the effort.
*
1-having potential utilizers assist in development
(Please write comments on opposite page
25.
B. Barriers to Utilization
Please rate the importance of the following barriers to utilization of
functional limitations concepts and scales, along with your confidence
in your response.
Importance/
Confidence
1. Counselor Concerns
a) fear of increased work load
b) fear of accountability
i) to supervisor and to other agency staff
ii) to client
c) suspicion about any new technique
d) fear that any failures or inadequacies will be
discovered and exposed
e) possible lack of proper training in utilization
of scales
f) fear of depersonalization of the counseling process
g) other (list and rate)
2. Possible Non-acceptance by Consumer Groups
a) fear of loss of a benefit
b) fear of further categorization
c) fear of over-emphasis on functional limitations
d) other (list and rate)
Question
Importance
Number
4
3'
2
1
Comments Concerning Page 25
1-a
2 5
-b-i
123
IV-B
-ii
25
-- Here, some of the "barriers" assume uses not inherent in a
-C - C
2
1
4
scale but in the way the organization uses the scale (counselor
- d
25
accountability, depersonalization of process, client eligi-
bility). Your user group questions asked about importance of
-e
3 4
a scale, given a use. You might also query about the relevance
-f
2
3
1
of a scale for particular uses. Some systems of client assess-
ment have only been implemented by assuring counselors that
2-a
223
the system was only for counselor's use in assessing client
gain, not as a system for assessing counselors (Arkansas,
-b b
232
Oklahoma).
- C
151
IV-B-1,2
- -04
/
-d*2
-- This is another area in which I have some difficulty in trying
to understand precisely how it should be answered. I think
-dx
in the development of any new system we are going to find
-d*4
varying degrees of all of the items and sub-items mentioned
in questions 1. and 2. However, I think that is par for the
course and I think these things should be anticipated, they
should be accepted that whatever system is devised should
take them into account and do the best we can with the negative factors that are to
occur. Rehabilitation has been operating on the same set of basic principles for
approximately the last 50 years or so. I think we have reached the point in time
where we are going to have to look for a different system. However, I don't think
that we should overthrow the old system until we have a new system that has a far
greater promise than the old one. I think we should have to put up with the flack
and negative attitudes and criticism, etc., that will come forward when any new
system is suggested. This should not be a basis for deciding not to go into this
type of operation. The measurement of functional limitations is far too important
an item to delay its exploration simply because some people by virtue of their
constitutional makeup will be opposed to it simply because it is a change from the
old system. I think if a workable functional limitations measurement system can be
developed and explained that it will be accepted by the leaders in the rehabilitation
field. They then will convince the others that this is the way we should go and
that we will eventually convert to some improvement over the present system.
IV-B-1
-- Found the 4 point scale difficult to use here. My ratings may be underestimating
counselor role as a barrier to utilization. The major barrier is the fact that
counselors may underestimate the skill required with their new system and not be
properly prepared. Trainers will need to be equipped to prepare counselors.
Generally, however, the four categories of barriers, counselor, consumer, agency,
RSA, do not include the major barrier, namely the public and the Legislature who
may be unwilling to make the financial commitment to restructuring the rehabili-
tation process.
IV-B-2
-- May have slightly overemphasized consumer role with exception of possibly client
resistance to evaluation time and psychological commitment.
IV-B-2-d
-- There is a strong feeling amont many consumers that bias will exist in measures
of function because criteria of limitation will be based on "ideal" non-disabled
people -- e.g., "the large, healthy, mentally capable adult in his/her late teens."
Disabled people wish to see maximum achievable function based on realistic dis-
abled role models, with criteria germane to specific attributes of the impairment.
*1 - fear that scale will not be relevant to their disability
*2 - disruption of disability-based lobbies
*3 - depending on scale, different coalitions will emerge in response
to the programmatic trade-offs implicit in any scale
*4 - client expenditure of time and effort in evaluation process
26.
Importance/
Confidence
3. State Agency Concerns
a) fear of increased reporting requirements
b) fear of exposure of State agency failures or
inadequacies
c) lack of training resources
d) change in bureaucracy takes time and staff
e) cost of change
f) fear of increased control by Federal government
g) different Federal programs have different eligi-
bility requirements. The functional limitations
scale(s) selected may not provide enough infor-
mation to determine eligibility for the VR program
or may not be able to determine eligibility for,
other Federal/State programs (e.g., similar benefits
such as Medicaid, Medicare, Title XX, etc.)
h) other (list and rate)
4. RSA Concerns
a) cost of change
b) change in a bureaucracy takes time and staff
c) the functional limitation scale(s) chosen may
conflict with RSA's Congressionally-mandated
definition of handicap or the definitions used
by other Agencies
d) how functional limitation scales would fit into
the eligibility process
e) other (list and rate)
Question Importance
Number 4 3 2 1
Comments Concerning Page 26
3-a
421
-b
2
41
IV-B-3
-c
16
15 1
-- Cost and training are to me the logical concerns, however,
-d
whether state agencies will recognize this before the fact
-e
-
16
is another question.
- -f f
1321
-- Most of these concerns aggregate to two: money and control.
-9
3211
Both are essential concerns to most state agencies. Neither
has prevented the implementation of program standards and,
4-a
2 4 /
therefore, both concerns could probably be mitigated. As
- -b
to ability to meet requirements for other state reporting,
1 4 2
if this is important, why wasn't it included in design
-C -
2 2 2
criteria?
- d
241
IB-B-3-g
- et 1
-- This point could also be turned about to indicate that
functional measures might be seen as threatening if they
were to be used as a common basis for eligibility, e.g.,
expanding numbers eligible for public programs.
-- This is an interesting point which warrants very close examination. I cannot speak
to it but hope other members of the advisory group will address it.
-- Of course this is one of the problems this approach tries to address. It is irrational
and counterproductive to define disability as "the inability to work" for purposes
of determining eligibility for such benefits as medical coverage. A thoroughly
researched FL inventory might not be the perfect solution, but it could hardly be
worse than what we're working with now.
IV-B-4
-- Again cost, but my concern is that RSA may not recognize the potential cost and
enter into a new system on the naive assumption that it will not be unduly costly.
IV-B-4-a
-- Change of size of eligible population, not merely organizational change.
IV-B-4_c
-- Since no one has fixed on one clear definition yet, this is not a problem
-- See B.3(g) above ("This is an
.).
IV-8-4-d
-- This is clearly a programmatic concern, less for the evaluation officer than for
the individuals concerned directly with client services and overall budget.
-- Reword - "Fear" F.L. scales would "straight-jacket" eligibility process.
IV-B-4-e
-- There is a fear that if left to States, some will use F.L. significantly, others
will not, and resulting disparity will result in sharply varying quality of services,
eligibility determinations, measurement of impact of services, and adversely affect
economy of scale in training, technical assistance, and research.
*1 - fear that without legislation of National consensus leadership, functional
limitations scales will be adopted non-uniformly leading to program disparities
27.
Potential Abuses
Please rate the importance and probability (likelihood) of the
following possible abuses of functional limitations scales, along
with your confidence in your response.
Importance/
Confidence
Probability/
Confidence
1. Denial of Services to clients who actually
should be receiving them
2. "Pigeonholing" of Clients into severity categories
a) functional limitations scales may lead to labeling
which could have negative consequences for the
client
b) reliance on functional limitation scales without
use of other relevant information may result in
inappropriate service delivery
3. Administration of Function Limitations Scales Without
Proper Training
a) by counselor
b) by support staff
c) others (persons not in the counseling situation,
e.g., employers, family members, etc.)
4. Using (interpreting) Results of Functional Limitations
Scales without proper training/understanding
a) by counselor
b) by support staff
c) by others
5. "Fudging" by Counselors
Question
Importance
Probability
Number
4 3 2 1
4321
Comments Concerning Page 27
/
413
151
2-a
323
25
IV-C
-b
511
2131
-- I don't think there is much more of a danger of
3-a
431
151
withholding services to clients under a new
-b
332
1411
system than there would be under the old system.
There will never be a fool-proof system and some
-c
53
I / 4
I
clients will always be short-circuited as unfor-
4-a
431
15 I
tunate as it may be. We currently pigeonhole
-b
332
1411
clients. We currently label clients. When it
comes down to the final analysis it is impossible
-c
-
422
133
for us to deal with clients without attaching
5
323
123
some kind of labels to the problems they have. I
really think the problem with labeling lies with
those of us in the professional field rather than
with the clients. I think in all probability the clients do a better job of pigeon-
holding and labeling us than we do of them. However, the fact remains we cannot
measure the extent of an individual's disabling condition unless we identify it and
know what it is. Once that process is undertaken, then the individual is labeled
and/or pigeonholed. It is a reality, a fact of life that we have to deal with. There
is no avoiding it.
-- Extremely difficult to project on this. Major concern is that a new system of eligi-
bility determination and client program planning will be introduced without adequate
training of counselors. Unless counselors are competent in the new system other
members of the agencies involved may be fooled into thinking they have a valid system.
-- The probability of potential abuse depends on several factors which are here un-
specified:
- The type of scale (objective, subjective);
- Its uses (accountability vs. plan development);
- The data gathering paradigm (self-report vs. measurement by professionals);
- The methods of introducing scales to users, etc.
Ratings are too risky given non-specification of these crucial parameters.
IV-C-1
-- Scale should obviously not exclude those eligible for service. However, scale
may determine eligibility; rather than conflict with eligibility criteria, it
may be an eligibility criterion.
IV-C-3-a
-- Likely to happen at least some of the time.
IV-C-3-a, 4-c
-- I doubt that I would favor making a functional assessment inventory available
for general use outside of the counseling situation with the possible exception
of trained and knowledgeable employers. We don't hand out our other psychometric
instruments for general consumption.
(Design team comment: We agree, but instruments often find their way into the
hands of unintended users).
IV-C-5
-- Again, it is difficult to assess any of this in the absence of the organizational
design to accompany the scales. If counselor performance is on the line, some
"fudging" is probable; if no sanctions impact on counselors, such behavior is less
probable.
-- Fudging is a danger primarily if it is necessary to disguise incompetence.
-- I think "fudging" will be possible with any inventory although it may be slightly
more difficult than it is now without one.
28.
Importance/
Confidence
Probability/
Confidence
6. Inappropriate Use of Scales
a) use of scale prior to its final development
b) use with population other than the one it was
designed for
c) other (list and rate)
Desirability/
Confidence
Feasibility/
Confidence
D. Other Uses of Functional Limitations Scales
Please rate the desirability and feasibility of each of
the other uses listed, along with a rating of your confi-
dence in your response
1. Cost-benefit/effectiveness studies
2. Development of National data base
3. Use by other service programs
4. Other (list and rate)
FUNCTIONAL LIMITATIONS
STATE-OF-THE-ART REVIEW PROJECT
A Delphi Study
Round 2,
June 1978
General Comments
-- Regarding the term "Functional Limitations." We have changed the name
of our inventory from "Functional Limitations Inventory" to "Functional
Assessment Inventory." This more neutral term fits both our philosophy
and item content more adequately.
-- All of my responses are based on the assumption that "functional
limitation scales" are more broadly conceived than many used at present.
My responses are with reference to scales that tap a wide range of roles
and tasks: vocational, social, self-care, mobility, etc. Functional
limitation scales that are limited to grooming, eating, toileting etc.,
I believe, are largely misplaced measures, after initial stages of
medical rehab. Relevance of functional capacities varies with time and
setting; I assume that the functional limitation scale applied at any
one point in time to a specific client is one that reflects the function-
ing needs of that client vis-a-vis his/her goals, expectations, resources,
environment, etc. Thus, "transportation functioning" is relevant
vis-a-vis job seeking but not when the client is a recently-injured patient.
I think the responses of other Delphi participants are largely formulated
in terms of specific types of scales they have in mind. This context
for our responses should be brought out into the open. (I note that
on "Comments Concerning Page 10," IIA3 comment #3, another respondent is
making a similar assumption.)
(Please write comments on opposite Dage
8.
II. BASIC ISSUES
Please read the comments from the previous Delphi carefully (pgs. 8-12)
as there appears to be a conflict between the previous ratings and the
respective comments. In this section, we are referring to user groups'
direct use of information provided by functional limitations assessment
instruments. Please remember also, that instrument developers must tailor
their instruments to the most important user group(s). We would like to be
able to identify that user group(s). Please rate the importance of functional
limitations assessment instruments for each of the following user-groups, and
your confidence in your response.
Importance/
Confidence
Client
Counselor
State Adminstrator
Federal Administrator
Researcher
Question
Importance
Number
Comments Concerning Page
8
4 3 2 1
II
Client
2 2 3
Counselor
6 1
State Ad.
} 6
Fed. Ad.
1 6
II - BASIC ISSUES
Researcher
5 2
-- Because of the emphasis on "direct" use, I
have changed my rating for client from 4/3
to 3/2. No doubt "clients" as a group are
much more diverse than the other users.
Hence, the emphasis on "direct" use will be
of importance to some, but not all clients.
This is a topic for research and testing.
-- While this information may be of limited use to clients, the ultimate respon-
sibility for determining eligibility lies with the conselor who is the most
important direct user of the information.
-- I continue to have problems rating these items, because I think instruments
relating to. functional limitations could be developed to be helpful for
most of these uses. However, there might be as many instruments as there
are uses listed.
(Design Team Comment: We agree. Perhaps we should clarify. There are many
possible user/groups, but some have more immediate needs for functional
limitations instruments than others. It is, therefore, important that we
first concentrate on designing an instrument for that group or groups.
Which one(s) of these groups are most important in terms of designing
initial instruments? Who has the most immediate need? Factors to consider
include meeting legislative intents, program/goals, reporting requirements,
etc.
-- Have maintained my rating of 3 for clients based on concept of direct use.
Use by counselors would exceed that of all other groups. Scale should be
designed primarily for counselor use. Headings of state and federal admini-
strator (singular) noted. I assumed you meant these to stand for the
headings used in II 3 & II 4 respectively.
(Please write comments on opposite page
9.
We are asking you to re-rate the possible uses for functional limitation
instruments. Please carefully read the comments from the previous round,
while keeping in mind we are interested in direct uses. Any item which has
been "blacked out" is one in which sufficient consensus has been attained.
They have, therefore, been deleted from this round.
1. Clients
Please rate the importance of the following possible direct
uses of functional limitations instruments by clients (refer
to comments on page 8 of round 1)
Impor tance/ Importance/
Confidence
a) Using such information to monitor his/her
progress throughout the rehabilitation
process
b) To gain realistic understanding of the effects
of their disability
c) Developing an Individualized Written Rehabilitation
Program with the counselor
Question Importance
Number 4 3 2 1
Comments Concerning Page 9
II-1-a
1
2
3
1
b
3 2 2
II-A-1
C
3 2 2
d
1
-- Again, I feel this information may be far too
technical for practical use by clients. I do
not mean to infer that clients can't understand
it nor do I think it should be withheld from them. However, depending
upon the complexity of the material, I think the ultimate real value of
such information is its usefulness to the counselor to translate this
material, into a practical form that a client can use may require more cost and
effort than the process is worth. The content and implications of all the
raw and processed data that may be necessary for such a system may best be
transmitted in a counseling situation.
-- Change in format helpful, but still feel that direct usefulness to client
is of limited value without extensive counselor interpretation.
II-A-1-d
-- I agree with whomever suggested this during Round 1. (eligibility determina-
tion)
(Design Team Comment: Functional limitations instruments have a definite
effect on clients regarding eligibility determination, but the client would
not use the results for this purpose, the counselor would. The importance
of the impact on the client concerning eligibility determination is an
extremely important one and will be emphasized in our final report. Eligi-
bility determination as an item more appropriately belongs under item 2
(counselor).
(Please write comments on opposite page
)
10.
Importance/
Confidence
2. Counselors
Please rate the importance of the following possible
purposes of functional limitation measurements for
counselors, and your confidence in your response.
a) fair and equitable eligibility determination
b) developing an individualized written rehabilitation
program with the client
c) measuring effectiveness of delivery of services
d) assessing client progress
e) counselor performance appraisal
f) weighted case closure
g) to minimize conflict between counselor and client
assessment of services
h) job placement activities
i) framework for organizing a comprehensive initial
interview
j) improved caseload management
k) Other (list and rate)
Note: "Professional Training and Continuing Education" was
mentioned as an additional category. However, the
design team decided to leave this under Section 6
-- relevant others
Question Importance
Number
4
3
2
1
Comments Concerning Page 10
II-A-2-a
-
-
-
5'
5 1 1
C
2 2 3
II-A-2-c
di
4 2
1
-- I don't get the full connection of the management
implication of this as an effectiveness measure,
el
3
3
1
except as it may be subordinate to the IWRP.
=4
6 1
A measure of functional limitations should lead
to the development of a realistic IWRP, and,
g
1 5
effectiveness of service delivery should be
hi
2 5
measured through the extent to which IWRP stated
expectations are met.
i
2
1
3
1
(Design Team Comment: One could view C as referring
j
3 4
to an evaluation of the effectiveness of the
services delivered under the IWRP through
successive use of a functional limitation instrument
revealing changes in the client's functional
limitations).
-- Changing the item to stress measurement should enhance its importance.
II-A-2-c,d
-- Although changes in functional limitations scores can be a useful measure of
progress in a medical rehabilitation setting, [ continue to doubt that such
scales are appropriate as measures in the progress OF Vocational Rehabilita-
tion. Some items may be the target of counseling intervention, eg. ability
to relate to other people, but many others would be entirely outside the realm.
Measures of client progress should focus directly on the rehabilitation goals.
II-A-2-e
-- Similarly, assessment of counselor performance should be concerned with out-
comes directly related to the counseling intervention. As indicated above,
I certainly would not assess counselor performance in terms of the amount
of change their clients showed on functional limitation measures.
Functional limitation scores could be relevant to counselor assessment in
at least two ways, however. They might be used to assess severity of
disability within a caseload so that counselors serving different populations
could be compared. They might also be useful if process rather than outcome
assessment is desired. Evaluators might look at whether services were pro-
vided consistent with client's limitations.
-- I don't see a functional limitation measure as a tool for use in counselor
performance appraisal.
(Design Team Comment: Just a note; we are interested in direct use, so we have
moved this item to 3 (State agency personnel). Also, thank you for your comment).
-- Have modified my rating from 4 to 3 since this makes it more consistent with what
I see are secondary uses (i.e., 2c through j) versus primary uses (i.e., 2a } b).
II-A-2-1
-- Interesting idea, but cannot see clearly how this would work.
-- I would like to see more explanation of the distinction between 2 "5" and the
2 "i". They seem the same to me; in VR the thorough diagnostic study is
supposed to be part of the IWRP process and hence is closely related to 2 "1".
(Design Team Comment: "i" refers to the initial meeting of the client with the
counselor. No rehabilitation plans are designed at this point. The IWRP is
developed at a later point in time in the rehabilitation process setting. Also,
as a note, the diagnostic study leads to eligibility determination. The IWRP
development occurs later following the establishment of eligibility).
(Please write comments on opposite page
)
11.
3. State Agency Personnel
Importance/
Confidence
Please rate the importance of the following possible
purposes of functional limitation scales for State agency
personnel, and your confidence in your response
a) estimating the population at risk*
b) setting long/short-range goals
c) program planning/evaluation
d) set priorities in program
e) budget justification
f) counselor performance appraisal
g) improved caseload management
h) to provide a common basis for dialogue between
all parties concerned (e.g., medical personnel,
evaluators, vocational counselors, etc.)
i) developing standards for providers of services
j) estimating community resources needed for meeting
the needs of the handicapped
k) other (list and rate)
*This is an indirect use of a scale. The design team
intended to illicit the idea of use of a scale to improve
the definition of severe handicap, so the population that
needs to be served can be counted. The definitional issue
is raised in Question 4-K
Question
Importance
Number
4 3 2 1
Comments Concerning Page 11
II-A-3-a
-
-
-
-
b
-
-
-
-
C
-
-
-
-
II-A-3-b and e
d 1 5 I
e
-
-
-
-
-- By your criteria this should not have been blocked
f
4
2
1
out, unless I misread the instructions.
g 2 3 1
h4
3
(Design Team Comment: You are correct. these two
i 2 1 3
questions should have been included in round 2.
j2 3 2
Please respond to these points in round 3).
II-A-3-d
-- I don't think this has any real value as a sole
criteria for priority setting except as aggregated data may reflect trends in
groups of elients coming to state agencies.
II-A-3-f
-- I don't see a functional limitation measure as a tool for use in counselor
performance appraisal.
-- Have modified my rating from 4 to 3, partly in response to other raters, and
partly because I do see this as slightly less important than 2e and 2g.
II-A-3-g
-- An obvious split in opinion here, but I see the use of FL scale(s) as
extremely useful in caseload management. Difference may be due to different
interpretations of the process of caseload management.
II-A-3-h
-- An accurate and realistic measure of functional limitations may serve as
the most important link between all of these groups as a common demonstration
for decision-making, provided the results of the measures are expressed in
language acceptable to all parties concerned.
-- Interesting point, but too diffuse to be rated highly.
(Design Team Comment: Refer to above comment for clarification)
II-A-3-i
-- This would require more explanation for me to rate.
(Please write comments on opposite page
)
12.
Importance/
Confidence
4. Federal Administrators
Please rate the importance of the following possible
purposes of functional limitation instruments for Federal
administrators, and your confidence in your response
a) estimating the population at risk*
b) program evaluation
c) set priorities in Federal program
d) develop programs to special target groups
e) provide basis for audit and program administrative
reviews
f) new agency reporting system
g) budget justification
h) establishing common program evaluation criteria
for all State agencies
i) initiate new legislation
j) identify research and training needs
k) to develop a new definition of handicap and severe
handicap
1) other (list and rate)
*This is an indirect use of an instrument. The design
team intended to illicit the idea of use of an instrument
to improve the definition of severe handicap, so the
population that needs to be served can be counted.
The definitional issue is raised in Question 4-K
Question Importance
Number
4 3 2 1
Comments Concerning Page 12
II-A-4-a - - - -
b
-
-
-
-
C 5 2
di
II-A-4-c
-
-
-
-
e 6 1
:
f
-
-
-
-
g
3
4
-- I don't think this has any real value as a sole
h
criteria for priority setting except as aggre-
-
-
-
-
i
gated data may reflect trends in groups of
-
-
-
-
j 4 2 1
clients coming to state agencies.
k 7
II-A-4-e
-- Despite one negative comment offered during
Round 1, I believe FL scale(s), adequately
developed, could serve a modest role here.
II-A-4-k
-- Aside from the obvious implication for determining eligibility for services,
development of a suitable definition of severe handicap may be the most valu-
able result of developing a system based on functional limitations rather
than categorical disabling conditions.
(Please write comments on opposite page
)
13.
5. Researchers
Importance/
Confidence
Please rate the importance of the following possible
purposes of functional limitation instruments for
researchers, and your confidence in your response
a) development of new techniques for and methods of
coping with or overcoming handicaps
b) develop new technological devices or adaptation
of existing devices to specific individuals
c) developing aggregate files of special target groups
d) utilization in demonstration and pilot projects
e) measuring impact of alternative case delivery and
administrative approaches
f) use as a control variable
g) other (list and rate)
Question Importance
Number 4 3 2 1
Comments Concerning Page 13
II-A-5-a
1321
b
1 1 4 1
C
-
-
-
-
d
-
-
-
-
II-A-5-a,b
e
-
-
-
f 3 1 2
-- FL scales may be useful for evaluating the impact
*g
1
of new techniques and devices, but do not contribute
to overcoming handicaps directly. For "evaluation"
use I would rate 5a and 5b as"3".
(Design Team Comment: We agree, and it has been
changed).
-- Functional Limitations Inventories will probably have indirect value in these
areas but lack sufficient specificity to provide new information for Research
and Development.
-- Have maintained my low rating of 2 in each. This is consistent with comments
made on Round 1 and with the concept of direct use.
II-A-5-f
-- Meaning is not clear.
-- Confusing -- it would make sense if it read (f) use as a dependent variable.
(Design Team Comment: We mean control variable, i.e., a variable for which
a researcher should control in the design of certain research studies, similar
to the way one would use demographic variables as control variables. The use
of functional limitation assessment instruments as a dependent variable is
implied in other items listed under II-5).
*1 - High specificity definition of outcomes or benefits that accrue to clients
(Please write comments on opposite page
)
14.
6. Relevant Others
Importance/
Confidence
Please rate the importance of functional limitation instru-
ments for the following relevant others in the
rehabilitation process and your confidence in your
response
a) trainers of rehabilitation personnel -- to focus
trainees' attention on client limitations rather
than their disability category
b) family members -- to increase supportiveness and to
improve the understanding of the disabled person's
problems
c) secondary service providers -- to focus their
attention on the needs of the client
d) other (list and rate)
Question Importance
Number 4 3 2 1
Comments Concerning Page 14
II-A-6-a
-
-
-
-
b
4 3
C 5 2
II-A-6-b
-- I feel we should give increased attention to
families, and if a Functional Limitations Inventory could help them better
understand a person's problems, I would use it. I suspect, however, that other
counseling techniques would be more effective.
II-A-6-b and C
-- The concept of direct use dictates a rating of no greater than 2 for both of
these.
II-A-6-c
-- Referral agencies need more specific, problem-oriented (and strength-oriented)
information about the client to help focus their services on attaining specific
measurable outcomes.
(Please write comments on opposite page
)
15.
B. Unique Client Populations
All respondents seemed to feel that all handicapping conditions must be
covered by functional limitations assessment instruments. Furthermore,
all of the identified client populations (including the following additional
populations cited: learning disabled, chronic pain patients especially those
with low back pain) present unique problems in the development of an instru-
ment (or instruments). Those developing such instruments should study each
of these populations to make certain that the limitations of these populations
are included in instruments which are developed. Failure to do so may result
in inequities such as denial of services to those who need them. The feasi-
bility ratings did vary which seems to indicate that feasibility is not
presently known. Feasibility will be determined as researchers explore these
problems. Finally, many of the comments were excellent. Most will be included
in the final report of this project.
Should you not agree with this statement or would like to make additional
comments, please do so on the opposite page.
We do feel it important to include Question 9, in a modified form. Please
answer the following question after reading the responses of the previous
Delphi, and giving careful consideration to the issues involved.
How should instrument developers progress in their work in regard to number
of scales to be developed?
Should try to develop 1 scale
Should try to develop more than 1 scale based on disability
type
For the immediate future, focus on developing more than 1
scale with an eye toward eventual development of 1 com-
prehensive scale
Other (list)
Question
Comments Concerning Page 15
II-B
Develop 1 scale
3
II-B
Develop more than 1
2
More than 1, then 1
-- I do not see the effort to develop
comprehensive
2
a comprehensive scale in the future
as contradictory to developing more
than 1 scale based on disability. I like the concept of a 'Branch' system as
a possible solution.
-- I don't think we should lock ourselves into a straight jacket at the front end
of this project. Let's allow research findings determine the ultimate form.
As I understand it what we want is a useful measure of functional limitation;
consequently, I think its ultimate form is subordinate to that concept.
-- Developing FL scales for separate disability groups would lead to non-comparable
sets of data: as if we're not all human beings directly comparable in our
strengths and problems in functioning. I would see nothing wrong with separate
Mobility scales or Communication or Self-care; but CP scales, MR scales,
epilepsy scales, hemi scales, etc. stem from a belief that pathology and
impairment define our focus, instead of role and task functioning. Some
functional limitations are disability specific (e.g., levels of lip reading
for deaf persons); but scales to cover these aspects might be developed as
addenda to the main scale. One unified scale is manageable through use of a
branched content structure and several levels of detail of measurement.
-- Would go along with an initial gross grouping of disabilities, e.g., physical,
mental retardation and mental illness, but believe further division by dis-
ability types would be impractical. Would prefer to measure at two/levels
(1) measurement of global levels for each specific domain (2) In-depth measure
for each domain identified as critical at level 1.
-- I still feel we should work toward a unified inventory, at least for all
physically disabled. The inventory may consist of several scales which can
be profiled a branched inventory, which could provide additional detail as
necessary for particular uses or disabilities is an interesting idea. Only
if a single instrument proves infeasible would I recommend that we develop
multiple inventories.
(Please write comments on opposite page
)
16.
C. Criteria for Functional Limitations Instruments
To be useful, functional limitation instruments must meet a variety of criteria.
Some criteria are more essential than others, and at times, some criteria are
not practical when measuring human qualities. Please rate the importance of
and feasibility for meeting each of the following criterion.
1. Measurement-Scoring Features
Consensus was reached concerning subscales (Question 1-c).
They appear to be important. Some comments indicated that
subscales would be especially useful in "obtaining detail
at a desired level and skip areas that are non-problematic
for a client" (see comments on Section II-C-4). Because
of the consensus, this question has been deleted.
There appeared to be a lack of comprehension of questions
a, b, and d as some results seemed to be in contradiction
to one another (i.e., did not logically flow). Please
Importance/
Confidence
Feasibility/
Confidence
review the comments from the last round and think about
the inter-relationships of the issues in a, b and d.
a) nominal, ordinal, equal interval scoring
Scoring of items on scales may be nominal
(descriptive only, cannot be ordered, or
added), ordinal (scaled items in ordered
sequence -- greater or lesser -- but items
not at equal intervals), or equal interval
(scaled items in order at numerically equal
distances on the property being measured.)
Please rate each of these criteria:
Nominal
Ordinal
Equal Interval
b) overall score of severity
c) use of subscales
Different user groups may need different levels of
specificity on information regarding functional
limitations. For example, the counselor may need
highly detailed information on client limitations,
while a researcher may need only a general indi-
caton of a class of behaviors (e.g., general
mobility abilities) while the State Administrator
might need only an overall score of severity. On
the other hand, some rehabilitation professionals
feel such scales should be brief, which may inhibit
the development of scales which are based on sub-
scales.
d) weighting items in relation to one another (keep
in mind the necessity of doing this for determining
a score of severity, and the decisions involved in
equating different types of limitations, e.g., how
do you weight physical limitations in relation to
mental limitations)?
Question
Importance
Feasibility
Number
4
3
2
1
4321
Comments Concerning Page 16
II-C-1-a
Nominal
5
1
5
1
Ordinal
2 4
15
Equal Int
3 1 2
5
1
II-C
b
4
1
3 2
C
-
-- I don't think we should lock ourselves
-
-
-
-
-
-
-
di
5
4 1
into a straight jacket at the front end
of this project. Let's allow research
findings to determine the ultimate form.
As I understand it what we want is a
useful measure of functional limitation; consequently, I think its ultimate
form is subordinate to that concept.
II-C-1
-- I agree with the comment that it is too early in the game to decide the measure-
ment answers, although these seem to be good questions. I also agree that
the result could be any or all combinations if several scales or a branching
system are used.
II-C
-- Some of the inconsistency in the results may have stemmed from different
interpretations of the concept of equal intervals. In the case of such
physical measures as weight, distance, size, etc. equal intervals may be
assumed with confidence, but in most measures which will be used in FL scale(s)
equal intervals can be no more than an expedient assumption. To sum over items
to obtain a total score, in practice, this assumption must be made. It will
be needed if results are transformed to obtain a severity score and will be
needed to weight items. We should strive for equal interval scale(s) despite
the difficulty involved.
II-C-1-a, nominal
-- Agree with the comments that ordinal scale is essential and that equal interval
would be desirable, but probably impossible.
II-C-1-a, equal interval
-- I think we can get an overall severity score without much difficulty simply
by totaling scores on the individual items or scales. I am fairly skeptical,
however, about what such a score would mean since identical totals could be
comprised of entirely different sets of items. Thinking of other instruments,
would a total of all of the scores on the GATB tell us anything about overall
aptitude or the sum of all MMPI scales be a useful measure of general
"craziness"? Still, a total may have some predictive validity even if it is
useless for counseling interpretation. If so, it may be worth using for such
purposes as defining severity of disability or caseload difficulty. This
can only be determined empirically.
II-C-1-b
-- This might be a very dangerous number, especially in a vocational, psycho-
social, or other non-medical context.
(Please write comments on opposite page
)
17.
2. Reliability - Both types of reliability (test-retest
and inter-rater reliability) were seen as important (based
on an analysis of ratings, confidence levels, and E-NE
responses) and hence has been deleted in this round of
the questionnaire.
3. Validity (does the item/scale really measure what
it is supposed to measure?)
The original purpose for including this item was our
concern over the quality of validity measures used in
validating instruments used in rehabilitation. We are
trying to determine the types of validity measures
which would be acceptable to the VR community. We have,
therefore, rewritten the rating system for these items.
The first column is a measure of the degree to which
you could find the measure acceptable as a measure of
validity. The second column refers to feasibility.
(Acceptability: 4 - Acceptable, one of the best measures
available; 3 - Acceptable, but should only be used if a
more acceptable measure is infeasible; 2 - unacceptable,
but may provide useful information if used in conjunction
with other measures - cannot be used alone; 1- unaccept-
able).
Acceptability/
Confidence
Feasibility/
Confidence
a) level of agreement with counselor assessment
b) level of agreement with other functional limi-
tation scales
c) level of agreement with client assessment
d) prediction of behavior in a wider range of set-
tings
e) should be left exclusively to the researcher
developing the instrument
f) other
QuestionAcceptibilityFeasibility
Number 4 3 2 1 4 3 2 1
Comments Concerning Page 17
II-C-3-a
141
24
b
4
2
6
C
3
4 2
II-C-3
3
d
142
5 1
2 2 1
131
-- I repeat my first round comment:
e
* f
1
validity measures will depend on what
1
*
f
1
1
statements are to be formulated
from FL scales, e.g., this is what
the client doesn't typically do,"
"this is what the client can't do,"
"this is what the client can't do in a specific environment, "this is what
the client can't do in a maximally supportive environment," etc. Validity
would also depend on how functional data are collected: interviewing client,
have client demonstrate functioning, etc.
-- None of these options, in my opinion, are appropriate for establishing validity.
For example how valid:
a. is counselor assessment?
b. are other functional limitations?
C. is client assessment?
What criteria would be used for comparison in a wide range of settings. (d)?
This should not be left exclusively to the discretion of the researcher.
Perhaps it could be left to the researcher and a group of knowledgeable people
in rehabilitation. (e)
II-C-3-e
-- Cannot answer this context with others. In practice, validity assessment
is an ongoing process which extends beyond the development phase. In
practice, validity assessment is usually viewed as a research function, but
rarely does a researcher act completely independently.
-- I am not sure how to respond to this. In practice, the researcher and the
provider of funding support must come to terms on an appropriate procedure
for establishing validity.
*1 - level of agreement with S.O.
*2 - Content validity
comments on opposite page
18.
4. Brevity/Detail of Scale
The results seems to be contradictory concerning the issue
OS brevity as both were rated highly. This may be be-
cause of an interest in the use of subscales or a branched
system (this might allow for brevity concerning items
not relevant to the client's condition, and detail on
those which are relevant).
These criteria are presented together as they bear a
direct relationship to one another. Brevity is often
mentioned as a desirable trait for functional limitation
scales as the user groups may be extremely busy and have
little time to spare. On the other hand, some profes-
sionals feel that detail on such a scales would provide
the counselor and client with important, specific infor-
mation helpful in setting up the clients' rehabilitation
plans. Also the more detailed the scale, the more it
Importance/
Confidence
Feasibility/
Confidence
forces the user of the scale to look at all of the specific
needs of the client. Please rate the following:
a) brevity
b) detail
c) this situation can be rectified by using a branch
system which has subscales which are or are not
used, depending upon the clients' circumstance
5. Objectivity - Subjectivity of Items
The results indicate that objectivity of items is highly
desirable and should be sought, but realistically, some
items may have to be somewhat subjective because of the
nature of what we are dealing with, and a desire for
brevity. The respondents noted, however, to strive for
objectivity.
6, 7,8. Predictiveness, Descriptiveness and
Prescriptiveness
Consensus was not reached on these questions. We are
therefore asking you to carefully read the comments from
the previous round and re-rate these items.
6. Predictiveness -- Prediction would provide the
users with information about a client's rehabilitation
potential. It would refine the eligibility criteria,
thereby insuring services are delivered to the target
population. The disadvantage is that particular strengths/
weaknesses of a given individual may not be sufficiently
accounted for. The measurement techniques may require
equal interval scales, or complex weighting.
Question Importance Feasibility
Number 4321 4 3 2 1
Comments Concerning Page 18
II-C-4-a 222
2 2 11
b
3 3
4 2
II-C-4-a
C 5 2
2 5
5
-
-
-
-
-
-
-
-- Again, it depends on the use. There are
E
22' 3
4
3
probably different versions needed for
different uses.
(Design Team Comment: The question has
been altered).
II-C-4-c
-- Branching (as well as development of modules) will lead to detail, with
brevity of data collection.
II-C-6,7,and 8
-- I think the Functional limitations measure should be basically descriptive
and prescriptive i.e. what is found wrong and what can be done to correct
it?. Predictiveness should be based on the expectation of both counselor and
client? I believe the motivational factor may destroy its effectiveness
as a predictive instrument together with attempts that may be made to use
it as a feasibility measure. Except for terminal illness, the clients
performance determines future actions.
II-C-6
-- I have marked Predictiveness "2" because I think the primary value of a
FL scale should be to identify impaired persons, not predict their success
in rehabilitation. Prediction requires the "other supportive information"
you have listed.
(Please write comments on opposite page
)
19.
7. Descriptiveness -- A descriptive scale would pro-
vice information for counselors in developing client plans
and in assessing client progress. The measurement tech-
Importance/
Confidence
Feasibility/
niques required for description would not be complex, but
Confidence
a problem may be the creation of a taxonomy with reliance
on semantics which may have limited utility:
8. Prescriptiveness -- This criteria has the advan-
tage of providing the users with information about
rehabilitation services which need to be provided (pre-
scription of services). This may reinforce decisions
made by rehabilitation professionals, but may cause
resentment because of "dehumanization" of the decision-
making process. It may also require the use of a lengthy
scale because of the possible number of prescriptions.
Ease of Interpretation and Training were seen as important
and feasible. Questions 9 and 10 were, therefore, deleted
from this round of the questionnaire.
Question
Importance
Feasibility
Number 4321 4321
Comments Concerning Page 19
II-C-7 241 331
-8 15 : 1 241
II-C-7
-- Lack of concensus on this item may be
a function at the ambiquity of your definition. My definition would not
entail the second sentence, which is not at all clear.
II-C-8
-- To prescribe services from functional limitation data assumes we know 'what
works' and also 'what works best' in addressing specific problems. FL data
may point toward service areas or potential options, but without tremendous
amounts of research, their use for prescribing is dangerous.
(Please write comments on opposite page
)
20.
III. COMPONENTS OF FUNCTIONAL LIMITATIONS INSTRUMENTS
A. Strict Functional Limitations Items
Consensus was reached on most items with all being seen as important com-
ponents. It was pointed out that "Health Limitations" and "Social
Attitudinal" categories are not true functional limitations. These
have been moved to Section B. Other items were added. Please rate the
desirability and feasibility of these items.
Desirability/
Confidence
Feasibility/
Confidence
1. Emotional (situational appropriateness)
2. Vocational Skills
3. Social Competence (interactions with other
people)
4. Homemaking/Child-care
B. Other Direct Supportive Information
There was some confusion over this Section and therefore will be repeated
after clarifying a few issues. First of all, the following items are not
functional limitation items. They are pieces of information which may or
may not be necessary or important to collect with functional limitation
information, because of their possible direct relevance. Furthermore,
the items listed will not necessarily be scaled items.
Importance/
Confidence
Feasibility/
Confidence
1. Exceptional Positive Personal Characteristics
(e.g., unusually attractive appearance to employer, ex-
tremely bright, extreme family supportiveness, wealth)
2. Exceptional Negative Personal Characteristics
(e.g., extreme ugliness, extreme unsociability, etc.)
3. Health Problems
4. Vocational History
Question
Desirability
Feasibility
Number
4
3
2
1
4
3
2
1
Comments Concerning Page 20
III-A-1
6 1
1 6
2 4 2
1
2 4 1
III-A
3
7
1 6
4
3
2
I
5
-- I believe the emotional and social compe-
*5
1
1
tence items qualify as functional limi-
Importance
Feasibility
tations if they are observed as relatively
III-B-1
3 2
1
3 2
1
consistent behavior of the individual.
2
2 2
2
32
1
The inclusions of specific vocational
3
5 1
4 2
and/or homemaking skills would decrease the
4
4 2
51
value of a functional limitation scale for
identifying and classifying certain client
target groups. The skills should be con-
sidered supportive information.
(Design Team note: We are dealing with functional limitations caused by an
interaction of a disability with other variables which prevents the carrying out
of activities. Disability can interact with other variables to limit vocational
and/or homemaking skills, so technically, it is a functional limitation).
III-A-1
-- I assume it is meant here that emotional functioning items would be stated in
terms of a person being able to assess situations and select appropriate behaviors
based on such assessments.
III-A-4
-- Does this refer to skills? If so, it would probably be appropriate only in
a branched inventory for individuals working toward a rehabilitation goal of
a homemaker.
III-B
-- The information can only be rated in importance when we know the purpose for
which it would be used. Assuming we are trying to describe employability I
am rating them 4. (The question is confusing. First you say the items are
not FL items, but in 7 it sounds as though we are again rating whether to
put Environment items in an FL scale.)
-- I have not rated these because, to me, their importance seems too much a
function of the specific use to which the FL scale is to be put. Many
other sets of data may be important, also. The important noint to be
made is that FL scales provide one of many possible sets of data, each
of which can be useful in decision- making to varying degrees.
-- I am rating these as all reasonably important and feasible to describe.
Certainly these are items of information most couselors would pay attention
to routinely. Since these are, would not be, incorporated into FL scale(s),
not sure why they are being considered here.
III-B-3
-- I assume this would have to do with absenteeism, etc.
*1 - educational
(Please write comments on opposite page
)
21.
Importance/
Confidence
Feasibility/
Confidence
5. Social-Attitudinal (acceptance of self and
others, motivation, etc.)
6. Demographic Information (age, sex, etc.)
7. Environment
Functioning is somewhat dependent upon environment, and
one might argue that a functional limitation scale should
measure client limitations within a given environment.
Also, realistically it must be remembered that a given
client functions in a variety of environments. There are
various issues involved in dealing with this topic. Please
rate the importance and feasibility of the following items
and options, after giving some thought to the potential
problems, and benefits of each
a) Exclude all environmental questions; they should be
picked up through the measurement of other variables
(if you agree with this statement check the follow-
ing box and move on to section IV).
b) Please rate the importance and feasibility of in-
cluding the following types of environmental factors
on functional limitations scales:
i) attitudes of others
ii) presence of technology relevant to the handi-
capped person's needs
iii) availability of services
iv) physical accessibility of home, work and
community
v) transportation
Importance
Question
Feasibility
Comments Concerning Page 21
43214321
Number
III-B-5
III-B-5
321
222
6 321
51
7a 3 boxes checked
-- I would rather see these things rated
in terms of the behavioral results.
-6-11-1111
12
-ii
111
21
Too much inference is required in rating
-iii
1
1
1
21
"acceptance of self" etc. What are
the behaviors that lead a counselor to
-iv
2
1
21
-v
1
1
1111
infer that a person does or does not
accept self? Let's try to observe and
record them instead.
III-B-7
-- Granted that the inventory would be conceptually "cleaner" without reference
to environmental variables, but I still believe it is essential that they be
included. I am greatly interested in the predictive validity of any Functional
Limitations systems which are developed, and I think prediction would be seriously
and probably fatally damaged if we looked only at the individual as if he/she
were functioning in a vacuum. Vocational rehabilitation is an interaction
between the person and the environment.
-- Environmental factors per se are not functional limitations. The clients'
ability to deal with the environment (i.e., 7C) is part of his/her functional
capacities.
(Please write comments on opposite page
)
22.
Importance/
Confidence
Feasibility/
Confidence
c) Please rate the importance and feasibility of
including items which measure the client's ability
to utilize or cope with environmental factors to
his advantage:
i) attitudes of others
ii) presence of technology relevant to the handi-
capped person's needs
iii) availability of service
iv) physical accessibility of home, work and
community
v) transportation
Question Importance Feasibility
Number 4 3 2 1 4 3 2 1
Comments Concerning Page 22
III-B-7-c
i 4
1 2 1
ii 4
2 2
III-B-7-c
iii 4
2 2
iv 4
2 2
-- I still see no point in rating the client's
V 4
2 2
ability to utilize these factors.
(Please write comments on opposite page
)
23.
IV. UTILIZATION
A. Facilitation of Utilization
Consensus was reached on most items. The results can be seen in the report
of the previous Delphi. Generally, the approaches listed were seen as ef-
fective, feasible and important. Three items need to be rated. The first
two were on the original Delphi and were rewritten while the third was added
by one of the panelists.
1. For Question one, assume that one of the main user
Probable
Effectiveness*/
Confidence
Feasibility/
Confidence
Importance/
Confidence
group of such instruments is the counselor, who will
use the results to develop an Individual Written Rehabili-
tation Program with the client.
a) Design of seminars/conferences to introduce
clients to functional limitations concepts and
instruments
b) Training counselors to teach clients on a one-to-
one basis, concerning the use and interpretation
of functional limitations information
2. Please rate the effectiveness of National Organizations
advocating the use of functional limitations concepts, the
feasibility of encouraging such organizations to do so,
and the importance of encouraging them
3. Having potential users assist in the development of
the instrument
*Use the definition of "desirability" in the front
of this Questionnaire for the definition of probable
effectiveness
Question
Prob. effec.
Feasibility
Importance
Number 4321 4321 4321
Comments Concerning Page 23
IV-A-1
-a231115 12212
-b 151 151
1231
2331
322
5 1 1
3241
34
-----
52
(Please write comments on opposite page
)
24.
B. Barriers to Utilization
There were some misunderstandings over what we were looking for here. Please
rate the likelihood (probability) of each occurring to a significant extent --
that is, one which will demand an active effort (i.e., time and/or money) if
it is to be overcome.
Likelihood/
Confidence
1. Counselor Concerns
a) fear of increased work load
b) fear of accountability
i) to supervisor and to other agency staff
ii) to client
c) suspicion about any new technique
d) fear that any failures or inadequacies will be
discovered and exposed
e) possible lack of proper training in utilization
of scales
f) fear of depersonalization of the counseling process
g) other (list and rate)
2. Possible Non-acceptance by Consumer Groups
a) fear of loss of a benefit
b) fear of further categorization
c) fear of over-emphasis on functional limitations
d) fear that scale will not be relevant to their
disability
e) disability based lobbies may fight change
f) client may fight change becuase of expenditure of
time and effort in evaluation process
g) other (list and rate)
Question Likelihood
Number 4 3 2 1
Comments Concerning Page 24
IV-B-l-a 2 4
b- 1 1
4
IV-B-2-f
- -ii 1
5
a 2 4
-- I don't think that use of a Functional Limitation
d 1 1 4
scale by the counselor will cost the client any
e 3 2 1
additional time or effort.
f 1 1 4
2-a 1 2 3
b 1 1 4
C ] 3 2
d 2 2 2
e 3 2 1
f
1 4 1
|
(Please write comments on opposite page
)
25.
Likel Likelihood/
Confidence
3. State Agency Concerns
a) fear of increased reporting requirements
b) fear of exposure of State agency failures or:
inadequacies
c) lack of training resources
d) change in bureaucracy takes time and staff
e) cost of change
i - organizational
ii - because of increase in eligible population
f) fear of increased control by Federal government
*g) different Federal programs have different eligi-
bility requirements. The functional limitations
scale(s) selected may not provide enough infor-
mation to determine eligibility for the VR program
or may not be able to determine eligibility for
other Federal/State programs (e.g., similar benefits
such as Medicaid, Medicare, Title II, etc.)
h) other (list and rate)
4. RSA Concerns
a) cost of change
i - organizational
ii - because of an increase in the ineligible
polulation
b) change in a bureaucracy takes time and staff
*This question was dropped as it is felt that many issues were
raised in one question, and the important issues were addressed
elsewhere
Question Likelihood
Number 4 3 21
Comments Concerning Page 25
IV-B-3-a 2 4
b 1
5
C 2 4
IV-B-3-e-i
d 1 5
e-i 1 4 1
-- It seems to me that a change in the method of defining
-ii 1 2 3
eligibility need not change the numbers of people
f
2
4
eligible for service. The degree of impairment required
g
-
-
-
-
to qualify can be altered regardless of the system used.
4-a-i 4 2
-ii|] 7 4
b
5 1
(Please write comments on opposite page
)
26.
Likel ihood/
Confidence
c) the functional limitation scale(s) chosen may con-
flict with RSA's Congressionally-mandated definition
of handicap or the definitions used by other Agencies
d) how functional limitation scales would fit into the
eligibility process
e) fear that without Federal legislation, functional
limitations instruments will be adopted by only some
States, and in different ways, which may lead to
disparities in the VR program
f) other (list and rate)
Question Likelihood
Number 4 3 2 1
Comments Concerning Page 26
IV-B-4-c 2 4
d
2 3 1
e
2 4
IV-B-4-c
-- Recommendations could be made to change the legislation.
IV-B-4-d
-- If they can be isolated, they should be the basis of eligibility.
IV-B-4-e
-Change the law to require its use.
(Design Team Comment: these were interpreted by us to be possible means to
overcome these barriers. Thank you for your ideas. They will be included
in the final report).
(Please write comments on opposite page
)
27.
C. Potential Abuses
In this Section, we are seeking opinion on potential abuses which are
likely to occur. It is impossible to tell which one in particular will
occur because of lack of information on type of scale, their uses, etc.
This requires the deletion of the probability category. We should, how-
ever, emphasize to persons developing and promoting the functional limitations
concept that certain potential dangers and abuses may occur, so that he/she
can take steps to prevent them.
Please rate the importance of the following items as potential abuses of
functional limitations assessment instruments (e.g., Is each abuse an
important one with which we should be concerned? To what degree?).
Importance/
Confidence
1. Denial of Services to clients who actually
should be receiving them
2. "Pigeonholing" of Clients into severity categories
a) functional limitations scales may lead to labeling
which could have negative consequences for the
client
b) reliance on functional limitation scales without
use of other relevant information may result in
inappropriate service delivery
3. Administration of Function Limitations Scales Without
Proper Training
a) by counselor
b) by support staff
c) others (persons not in the counseling situation,
e.g., employers, family members, etc.)
4. Using (interpreting) Results of Functional Limitations
Scales without proper training/understanding
a) by counselor
b) by support staff
c) by others
5. "Fudging" by Counselors
Question Importance
Number 4 3 2 1
Comments Concerning Page 27
IV-C-1 4 1 2
2-a 1 4 2
-b 4 3
3-a12 5
-b - 2 4 1
-C 3 2 2
4-
a
3
4
-b
3
3
1
-C 3 2 2
5
4 3
(Please write comments on opposite page
)
28.
Importance/
Confidence
6. Inappropriate Use of Scales
a) use of scale prior to its final development
b) use with population other than the one it was
designed for
c) other (list and rate)
Desirability/
Confidence
Feasibility/
Confidence
D. Other Uses of Functional Limitations Scales
All items were seen as highly desirable and feasible.
1. Cost-benefit/effectiveness studies
2. Development of National data base
3. Use by other service programs to determine eligibility
4. Other (list and rate)
Question Importance
Number : 4 3 2 1
Comments Concerning Page 28
-b 3 4
FUNCTIONAL LIMITATIONS
STATE-OF-THE-ART REVIEW PROJECT
A Delphi Study
Round 3
June 1978
General Comments
The general comment that relevance of functional capacity varies
with time and setting is particularly appropriate. The final report
and conference should address this point.
I am defining "direct use" as nothing or no one in-between" the user
of the information produced by the scale or inventory and the initial
compilation of the information at whatever aggregate level (an indi-
vidual case, an agency caseload, a counselor's caseload, etc.) By my
use, new direct use occurs at the initial aggregation of the primary
data about a person. A client and counselor are both direct users.
So are quality assurance specialists, Federal and State administrators,
and other specialists. Secondary users work with the products of these
direct users.
The instrument is a tool to be used under prescribed conditions. The
strictest criteria should apply to assuring objective measurement of
function. The measurement task is separate from use. The first
assumption should be that the primary purpose of FL inventories is
for client-counselor joint decision-making. Subsequent users adapt
or improve the basic information used at the counselor-client level
by identifying aggregate uses. A focus for research should be to
identify obstacles to such subsequent, direct uses, as well as indirect
uses, and explain how to mitigate these peoblems.
1 - It appears that "direct use" -- as commented upon on page 8 of the
comments on Round I, and page 13 of Round II, has confused people.
This is the reason for my comments on the point. I would like to
see this discussed in some detail at the conference.
Please write comments on opposite page
8.
II. BASIC ISSUES
A) We have examined the results of the first two rounds and we are
still not sure that we understand the reasons for some of the
responses. We are going to ask you once again to rate these user
groups noting that we are interested in direct uses of functional
limitations instruments. Some persons have indicated that all
user groups listed here are important. We agree. However, we must
set some priority user groups so that instrument developers can
work on instruments for these "high priority" user grouns.
Please re-rate this section. In addition, PLEASE PROVIDE AN
EXPLANATORY NOTE FOR EACH OF YOUR RATINGS, as this will help us
to understand why certain user groups were viewed as more important
than others.
Importance/
Confidence
1. Client
(explanation)
2. Counselor
(explanation)
3. State Administrator (State Administrator,
Counselor Supervisor, program evaluator, advisory
and policy making boards, commissions, etc.)
(explanation)
Comments concerning page 8
Question
Importance
Number
4
3
2
1
II-A-1
3
3
3
-2
9
II-A (General)
-3
1
6
2
-- It seems that there is quite a bit of confusion
surrounding the interpretation as to what direct
uses that may be made of a listing of the functional
limitations exhibited by clients. This is especially
true in relation to clients, counselors, state administrators and other related categories
of administrative support personnel. My basic assumption for all of these groups of people
is as follows:
The counselor is the primary person who will make the most direct use of any information
pertaining to any client's functional limitations. This, I think, is absolutely essential
as a basis for determining eligibility and feasibility for rehabilitation services. While
the client may be entitled to share in this information and partake of it and certainly
appropriate information should not be withheld from the client. The primary direct user
of this information is the counselor who is making the determination of eligibility and
feasibility for rehabilitation services. If functional limitations are used in the manner
in which I envision them as a basis for eligibility and feasibility, it appears there will
be a considerable amount of highly detailed technical information that would be of real
value only to the counselor and the medical, vocational, psychological and other categories
of staff relating to a given client's case. I can compare this situation to that of me
going to my personal physician for a diagnosis of a given condition on the assumption that
the physician in the process of arriving at the diagnosis may use a variety of different
test procedures and processes to arrive at a decision. While I would be interested- in
the results of all of these procedures, it is highly questionable as to whether or not I
would have any significant understanding of the results of the processes as they would
be reported to the physician. For example, I would not understand a reading given on a
lab report as to a white blood count, however, I would want to know from the physician
something about the extent of my condition in relation to the white blood count: I see
a similar process involved in clients getting highly detailed and technical information
from their counselors on functional limitations. I am also aware that physicians are
required to report to various health agencies the number and kinds of diseases they have
treated and diagnosed during any particular time period. While I as an individual may
have some general interest in that information, I would not be specifically concerned
about individual cases. I would consider this similar to the State agency administrator
and other related administrative support staff as far as functional limitations are con-
cerned. This information should be available in aggregated form and will be of value to
administrators, researchers, evaluators, advisory councils, etc., in aggregated form.
but should not be of specific client centered interest to this group of people. The
primary direct user of information pertaining to functional limitations must be the
counselor and secondly the client as is interpreted by the counselor. All other
interested individuals should have access to the information and should make use of it
to determine trends, future directions, efficiency effectiveness and impact of programs,
some determination as to whether or not an adequate penetration of the handicapped podu-
lation is being made and so forth. I make this as a blanket statement covering all items
listed under II a) 1. Client, 2. Counselor, 3. State Administrator.
I have marked the items on page 9 II 4. and II5. in a similar manner with the same
explanation applying to them.
One additional note, I feel researchers should have a vital interest in all information
it listed about functional limitations so that we can have some assistance in charging
the course that we must take in developing rehab programs centered around functional
limitations. I think the researcher's role in this is vital because we will need factual
data based upon competent knowledge rather than on beliefs and assumptions.
-- Client and Counselors. Based on the above comments, note that "providing the frame-
work for initial interview" is only slightly important for the "client", but is very
important for the counselor. To gain a realistic view of limitations is very important
for the client, but is also important for the counselor. On a four-point importance scale,
most elements would score within one point of each other for both clients and counselors.
-- My ratings can be explained as a group: of highest and equal importance are the uses
of FL scales (1) for individualized planning and feedback re. client progress and (2) for
researchers to have sensitive measures of rehabilitation outcome. The management and
evaluation concerns of administrators, I believe, are a bit less important. I believe the
use, re. the client-counselor dyad are most important because good FL scales will assist to
enhance directly rehabilitation planning and outcomes: the most direct benefits of any use.
Uses for researchers are also rated high because I believe the greatest long-term benefits
of good FL scales will be attained through that use.
Comments concerning page 8 (b)
-- I am interpreting direct use to mean that the client will see or hear his/her scores
on the scale(s) and from that information be able to use them for purpose of decision
making (i.e., in vocational choice, rehab plan), increasing understanding and/or self-
awareness, acceptance of the rehab plan and services entailed and finally, possible
acceptance of the disability itself. With absolutely no input from the counselor other
than bare scores on a profile, use would be minimal. Therefore it is difficult for me
to identify direct and independent use by the client. For some it may be high. others
quite low. Thus, the intermediate rating.
-- In my opinion, the lowest priority application for a functional limitations inventory
is direct use by clients. As explained in earlier rounds, I do not see the inventory as
an appropriate measure of progress in rehabilitation, but it may help clients to under-
stand their conditions and participate in decision-making.
-- Joint involvement of client and counselor must be based on accurate and adequate
communicator about the client. Functional assessment is a tool for this job. The
design team is wrong in saying assessments have no direct use for the client in eligi-
bility matters. The client has a right and need to know and comprehend assessments
of his disability used in determining eligibility. His access to FL inventories for
this purpose is a direct use of the data.
-- Even though you have explained the "direct use" implications from your perspective,
a client aware of the workings of the system may in fact directly use the measure -- if,
for instance, it is to be used in formulating the IWRP, the client should be as direct
a user as the counselor.
-- Client "active" usage of the instrument would only be a secondary gain to the primary
usage of the counselor.
II-A-2 (Counselors)
-- Counselor would gain a more comprehensive and uniform understanding of his client's
status. Areas of strength and need can be more precisely pinpointed and ameliorated
without extensive reworking at each counseling session.
-- For assessment, development of plan.
-- This is the primary user and scale(s) should be designed for his/her clinical use
even at expense of utility for others. Scale(s) should be designed for decision making
primarily in eligibility determination and client rehab program planning. Other uses
are secondary.
-- Counselors and researchers are the highest priority groups. For counselors, a
functional limitations inventory should help especially with eligibility determination
and planning with individual clients.
-- Counselor initiate FL assessments for clients, share the results with clients, work
with the client to set objectives and goals commensurate with joint counselor-client
needs assessment.
II-A-3 (State Administrators)
-- Direct use of scores, summarized and treated statistically, can be made primarily
for planning and evaluation (program and personnel).
-- Here, I am assuming use of summary measures primarily -- for reporting, budget,
performance reviews, etc.
-- I continue to give 4's to the categories State Administrator and Federal Administrator
because I think it is as important and probably more feasible to use FL instruments to
assist with program decisions as it is to use them to assist counselors in making
decisions regarding individual clients. Even if this is considered an "indirect" use of
FL instruments, it is still a primary reason for developing such instruments.
Comments concerning page 8 (c)
-- Again usage by State Administrators would only be a secondary gain to the counselor.
This next closest link to the counselor could use the instrument ratings in close
conjunction with closure stats to better understand the client-counselor process.
-- A functional limitations approach should aid administrators in making decisions
which require detailed knowledge of the population needing services, i.e., how many
people, with what kinds and degrees of problems, needing what sort of assistance?
-- Quality assurance efforts, and moves to increase the independing functioning of
counselors while emphasizing monitoring of outcomes, are consistent with efforts to
develop improved FL scales.
Please write comments on opposite page
)
9.
II. BASIC ISSUES
Importance/
Confidence
4. Federal Administrator (Commissionor, Policy
and advisory boards, commissions, etc., and
persons involved in program operations, planning,
evaluating, etc)
(explanation)
5. Researcher
(explanation)
Comments concerning page 9
Question
Importance
Number
4
3
2
1
II-A-4
2
3
4
-5
6
3
II-A-4 (Federal Administrator)
-- Probably, program visibility is not to be expected at
this level of detail.
-- Direct use of value primarily in program planning and
evaluation.
-- The level of removal from the real operation of usage of the scale would possibly only
allow usage of the instrument results in close conjunction with a lot of documenting
evidence to understand the process.
-- Federal officials seek to raise the common denominator of rehabilitation assessment to
higher levels of accuracy so that national standards can apply. Increased confidence that
similar measures are being used in controlled circumstances enhances the claims of Federal
administrators in expressing program policies and accomplishments.
-- Another high priority group. Especially pressing needs related to functional limitations
include developing a new definition of handicap and severe handicap, identifying research
and training needs and needs for new legislation.
II-A-5 (Researcher)
-- A valid scale -- and development and testing of such a scale -- will call for more
research -- and hopefully also, contribute to solution of current research questions.
Please see comments on research in round 1 also.
-- Quite important use, second only to counseior. (I) stress that scales should be
designed for counselor use in eligibility determination and client program planning.
Other uses by counselor and by others should be considered in considering the cost
and significance of the scale, but should be seen as secondary purposes. Serious
restrictions should be set on the scope of uses and users. Without such direction,
primary uses will not be effectively realized. This viewpoint has caused me to be
somewhat more conservative in my ratings of uses other than the primary 2 for the
counselor.
-- The researcher could use the scale to more precisely monitor the process of rehab
and its summary factors.
-- Technological, medical, social, environmental and other change areas require
continual research and monitoring to keep FL inventories current. In addition, researchers
will use the scales in innovative ways to demonstrate therapeutic possibilities, etc. The
contributions of researchers, and the importance of FL for them, is second only to direct
use of counselors/clients.
( Please write comments on opposite page
)
10.
We are asking you to re-rate the possible uses for functional limitation
instruments. Please carefully read the comments from the previous round,
while keeping in mind we are interested in direct uses, Please rate
each item independently (i.e. responses do not have to be consistent
with responses to other questions).
1. Clients
Please rate the importance of the following possible direct uses of
functional limitations instruments by clients. We have identified
some pros and cons to keep in mind as you respond to these items.
Pros
1. facilitate goal setting
2. gives actual point value to progress in gaining greater functional
capacity
3. more scientific , precise and objective than self/counselor
ratings.
4. could serve to reinforce his/her confidence
5. may be motivational.
Cons
1. unnecessary as monitoring is already done by others (e.g. Counselors/
providers)
2. may be too technical
3. may be too time-consuming
4. possible preoccupation with instrument's points rather than actual
progress
Importance/
Confidence
a) Using such information to monitor his/her
progress throughout the rehabilitation
process
b) To gain realistic understanding of the
effects of their disability
C) Developing an Individualized Written
Rehabilitation Program with the counselor
Comments concerning page 10
Question
Importance
Number
4
3
2
1
II-A-1-a
3
5
1
-b
2
4
2
1
II-A-1-a,b,c
-C
4
4
1
*-d
1
-- The most critical "con" is not listed. That is,
many items in a comprehensive functional limitations
inventory are not subject to change as the result of
rehabilitation services - they are simply descriptors
of the client's condition. Measures of progress need
to be directly related to treatment goals.
-- The pro-con arguments show goal setting, client understanding, and the IWRP as primarily
counselor concerns. Monitoring progress, understanding of evaluation, and seeing the basis
for outcome-oriented intermediate steps in an IWRP are seen as "professional" concerns for
the counselor
not to be duplicated by the client or bothered with by him. This is
very wrong! Goal-setting must be joint. See Dyer and Vriend, "A Goal-Setting Checklist
for Counselors," Personnel and Guidance Journal, April 1977. It is a vital part of rehabili-
tation for the client to learn to set goals, measure progress, and use objective measures
of goal attainment. The client should assume more and more direct responsibility based on
knowledge. Understanding will be limited for some clients. But the principle of direct
use of FL assessment by clients is very important.
-- Ratings (3 and 4) are based on my assumption that the FL scale in question will be
developed in a language that clients can directly comprehend, avoiding jargon. I also
assume that enhanced client participation in planning and monitoring progress will enhance
the rehabilitation process and outcome.
-- I continue to feel the direct use of an FL assessement instrument by the client would
be low, but I acknowledge that it might be more important if future clients are served in
programs leading to increased independent living.
-- See comments on previous page re. development of the IWRP.
-- Note: I have re-entered "eligibility as a concern. I feel that, since the measure
might be used by the system for eligibility determination (either for all services, or
for some particular programs), the client/applicant is a "user."
-- If reliable and valid functional limitation scales can be developed for direct client
use, they would assist the client in: 1) developing an accurate understanding of the effects
of an impairment; 2) understanding and accepting eligibility decisions; 3) participating
in rehabilitation planning; 4) assessing progress; and, 5) identifying needs for further
assistance.
II-A-1-a
-- Probability of constructing a scale sufficiently reliable to accurately measure changes
over time plus having clients who are able to assimilate information is low.
II-A-1-b
-- I see some merit in use of scale in helping client understand the relation between the
disability and its functional impact, but see no need to ascribe to this purpose if it
required any modification of the scale which would lessen its clinical utility to counselor.
II-A-1-g
-- Potentially useful for many, but not all, clients.
*eligibility determination
( Please write comments on opposite page
)
11.
Importance/
Confidence
2. Counselors
Please rate the importance of the following possible
purposes of functional limitation measurements for
counselors, and your confidence in your response.
a) framework for organizing a comprehensive
initial interview
I
b) fair and equitable eligibility determination
c) developing an individualized written rehabili-
I
tation program with the client
d) measuring effectiveness of delivery of
services
I
e) assessing client progress
f) to minimize conflict between counselor and
NI
client assessment of services
I
g) job placement activities
h) improved caseload management
**i) counselor performance appraisal
**j) weighted case closure
k) Other (list and rate)
*Note: Weighted case closure and counselor performance
appraisal were deleted here as we are referring
to direct use. They therefore, more appropriately
belong, and have been moved to, Section 3
(State Agency Personnel).
Comments concerning page 11
Question
Importance
Number
4
3
2
1
II-A-2-a
3
1
4
1
-d
2
4
2
-h
3
5
II-A-2 (General)
*-K
1
-- I would add "To have the client gain a realistic
understanding of the effects of their disability"
as an important purpose for counselors. This is
similar to "minimize conflict between counselors and clients," but as a product of the
action, not the action itself. Gaining realistic understanding enhances client ability
to do the IWRP. Since the IWRP is a joint process, using FL inventories to increase client
understanding of his abilities and limits is a direct benefit to counselors. I like listing
of the items to appropriate the rehab process.
-- Functional limitation measures represent an assessment tool, not an organizational frame-
work.
-- Variability in response to this item may be due to different perspectives as to how the
scale relates to the interview, to the meaning of the word "framework", etc. Since we are
talking about the first interview, no prior scores are assumed to be available. Some scale
information may be obtained by the counselor at that time, but most scale information will
be obtained later. Direct use of the scale here is seen as minimal.
-- At this point I think it is irrelevant as to whether or not a report of functional
limitations can be used as the basis or framework for organizing a comprehensive initial
interview. Unless I am badly mistaken in the sequence of events that I envision occurring
in the use of functional limitations, an initial interview would come long in the process
before an attempt was made to gather information pertaining to functional limitations.
Unless I am missing a point somewhere it would appear to me that the initial interview is
made after referral is made to the agency and the client and agency representative get
together. I do not think it is possible to have any kind of information pertaining to
functional limitations at that point in time. If the assumption is being made that the
purpose of the initial interview is to gather information pertaining to functional limi-
tations then I think the amount of information that might be obtained from the counselor
specifically in relation to functional limitations would be minimal. However, I do think
that some value might be gained from using an intake procedure as a basis for acquiring
information pertaining to functional limitations at a later date.
II-A-2-d
-- FL assessment information to be used for this purpose would need to be translated (from
raw data to an evaluation statement). This use, then, is not direct, although I believe
it is important.
-- If you mean the effectiveness of delivery of all services rendered to one client by the
counselor, there is no way in which a casual relation between delivery of services (inde-
pendent variable) and decrease in functional limitations (dependent variable) could be
inferred. If you are referring to counselor being able to generalize over his total case-
load concerning the relation of these two variables, that is a possible use but an unlikely
one in practice. Relation of service to outcome as measured by the scale would be more
logically assessed by the researcher or agency program evaluator.
II-A-2-h
-- Potential use, but only after scale developed for primary uses.
-- Improved caseload management probably could be the result of reliable information per-
taining to functional limitations. However, again, I think it is far too theoretical at
this point in this procedure to make any significant determination as to what extent case-
load management could be improved through the use of functional limitations. I think this
is an item that should be regarded as having potential importance but the ultimate impact
of it on the rehabilitation community in general should be a result of a long process of
very thorough research as to whether or not basing eligibility determination on functional
limitations actually does make a significant difference in the size and manageability of
the counselor caseloads.
-- Caseload management requires other skills and techniques (e.g., efficient use of time,
prompt follow-up, records management, caseload review, etc.)
*determine independent living services
( Please write comments on opposite page
12.
3. State Agency Personnel
Importance/
Confidence
Please rate the importance of the following possible
purposes of functional limitation scales for State agency
personnel, and your confidence in your response
I
a) estimating the population at risk*
b) setting long/short-range goals
I
c) program planning/evaluation
I
d) set priorities in program
e) budget justification
**f) counselor performance appraisal
I
g) improved caseload management
h) to provide a common basis for dialogue between
all parties concerned (e.g., medical personnel,
evaluators, vocational counselors, etc.)
i) developing standards governing rehabilitation
facilities and other providers of services used
by the rehabilitation agency
j) estimating the need for workshops, rehabilitation
and other facilities and services within the state
to meet effectively the needs of the handicapped
k) weighted case closure
i) other (list and rate)
*This is an indirect use of a scale. The design team
intended to illicit the idea of use of a scale to improve
the definition of severe handicap, so the population that
needs to be served can be counted. The definitional issue
is raised in Question 4-K
**This is not a direct use, rather it would be used in
weighted case closure (k) which could be used for
counselor performance appraisal
Comments concerning page 12
Question
Importance
Number
4
3
2
1
II-A-3-6
1
2
3
-e
1
2
4
-h
4
3
1
1
II-A-3 (General)
-i
1
5
2
-j
1
5
3
-- Note: On weighting: in the long-run, such scales
-k
5
3
could undoubtably contribute to "weighting," to
allow consideration of more "severe" cases on an
equal footing with those easier to close. But in
the short run, the target in this concern will
probably be researchers, not administrators.
II-8-3-b,e,i,j
-- Laudable purposes and probably the scale(s) could contribute. To repeat, such uses of
the scale should come only after it has been designed for primary uses.
II-A-3-6
-- Whether it is desirable to use this as a basis of setting long and short-range goals,
I think this is an ancillary or subordinate use of the functional limitations processes
as I have interpreted them on the basis of this project. I don't think that a program's
long and short-range goals can be set entirely on. the use of functional limitations. They
must be taken into consideration in developing such goals, however, they should not be
the sole basis for it.
II-A-3-c
-- Aggregate data reflecting the severity of the functional limitations in the total case-
load might help.
-- Budget justification - I think this is an important item. Money is always important.
Budgets could be based upon services known to be needed as a result of an analysis of
clients that are made eligibile on the basis of functional limitations. I think this has
far-reaching implications for funding the rehabilitation programs.
II-A-3-h
-- I have raised my rating to 3+ (or 4) because I see this as an extension of the counselor's
clinical function.
-- I think it is extremely important that functional limitations be expressed in such a
manner that there is a common language between the categories of personnel listed in this
item. I further feel that one of the problems we have always had in the rehabilitation
field in trying to deal with the extent of impairment that any client may exhibit lies in
the incapacity of the various disciplines within the field to adequately communicate be-
tween themselves. It may be that if some common language can be developed in functional
limitations that will meet the technical and professional needs of the different disciplines,
this limitations. may be one of the greatest breakthroughs that could be made with the use of functional
-- A rating of 4 is based on the assumption that the FL scale will be a non-jargon, non-
technical instrument; this would facilitate communication of persons with different
processional languages and training.
II-A-3-i
-- Since it is anticiapted that in the future with the emphasis on the severely disabled
and especially with the new legislative thrust regarding comprehensive services and inde-
pending living, standards must be developed for facilities and providers of services used
in the rehabilitation field. These same comments would apply to Item j. At this point it
may appear that I am contradicting myself by saying that these items are not particularly
important in setting long and short-range goals, but yet emphasize the importance of items
h, i, and j and especially i and j. It is my impression that long and short-range goals
cannot be established until the activities described in items h, i, and j can be put into
operation in the rehabilitation field, then goals may be established using the information
generated from these procedures.
Comments concerning page 12 (b)
-- Might be a useful component in a prospective reimbursement system.
II-A-3-j
-- May assist in estimating the demand for such things as extended employment.
-- In that functional limitations of any given type can often be responded to in numerous
ways (with a mixture of services), use of FL scales for estimating needs seems to me to be
based on a faulty premise: that functional limitation X implies need for service x', a
short-sighted approach to service planning.
II-A-3-k
-- I feel it is desirable to take this into consideration, but this should be an item
subordinate to the total direction that development of a concept of functional limitations
must take. It may be that at some point in the future it is not desirable to use weighted
case closures as a measure of counselor performance. It may be that functional limitations
themselves will become inherently weighted case closures. It may be that future legislation
will restrict or expand the extent to which clients can be served on the basis of functional
limitations. Again, I feel that weighted case closures is subordinate to the total picture
and this item should be held in abeyance until we know more about the impact of functional
limitations on the rehabilitation community.
-- This is of some recognized importance and can be achieved readily after the scale has
been designed for counselor use.
( Please write comments on opposite pade
)
13.
4. Federal Administrators
Importance/
Confidence
Please rate the importance of the following possible
purposes of functional limitation instruments for Federal
administrators, and your confidence in your response
a) estimating the population at risk*
I
b) program evaluation
c) to assist in setting priorities in Federal programs
I
d) develop programs to special target groups
e) provide basis for audit and program administrative
I
reviews
I
f) new agency reporting system
g) budget justification
h) establishing common program evaluation criteria
I
for all State agencies
I
i) initiate new legislation
I
j) identify research and training needs
**k) to develop a more precise definition of handicap
I
and severe handicap
1) other (list and rate)
*This is an indirect use of an instrument. The design
team intended to illicit the idea of use of an instrument
to improve the definition of severe handicap, so the
population that needs to be served can be counted.
The definitional issue is raised in Question 4-K
**Consensus was reached on this item, but we are exchanging
"more precise" for "new". If there are objections, please
make them known in your comments
Comments concerning page 13
Question
Importance
Number
4
3
2
1
II-A-4-c
4
1
-g
125
II-A-4-c,q
-- These items are slightly important because
functional limitations scales produce "micro"
data whereas budget and priority decisions are "macro" level, e.g., broad groupings such
as health, manpower, etc. (Items C and 4-g).
(j and 4-k): National leadership is essential to initiate new legislation of a broad re-
search function. Fact is that Washington is where the big decisions are made on these
points.
II-A-4-c
-- Aggregate data derived from functional limitation scales may be useful in assessing
the utility of impairments in setting Federal program priorities.
II-A-4-g
-- Aggregate data reflecting the severity of the functional limitations in the total case-
load might help.
-- Budget justiciation - I think this is an important item, money is always important.
Budgets could be based upon services known to be needed as a result of an analysis of
clients that are made eligible on the basis of functional limitations. I think that has
far-reaching implications for funding the rehabilitation programs.
I Please write comments on opposite pade
)
14.
5. Researchers
Importance/
Confidence
Please rate the importance of the following possible
purposes of functional limitation instruments for
researchers, and your confidence in your response
a) evaluating new techniques for and methods of coping
with or overcoming handicaps
b) evaluating new technological devices or adaptation
of existing devices to specific individuals
I
c) developing aggregate files of special target groups
I
d) utilization in demonstration and pilot projects
e) measuring impact of alternative case delivery and
I
administrative approaches
f) use as a control variable*
g) develop a data base from which researchers could
determine new areas of research and technology to
be pursued
h) other (list and rate)
*For purposes of explanation, we repeat a comment from the
first round "In many instances when randomization is not
possible in studies, say, of effectiveness of a treatment
measured by such variables as change in income, client
satisfaction, etc., a measure of functional limitation(s)
may be more useful than type of disability or other
demographic variables as a control"
Comments concerning page 14
Importance
Question
Number
4
3
2
1
II-A-5-a
3
5
1
-b
1
5
3
II-A-5-a,b,f,g
-f
5
3
-g
2
5
1
-- My feeling on all of these items is essentially
the same. Without question, there must be a
thorough research effort developed at the same
time that any attempt is made to use functional
limitations as the basis for eligibility and feasibility for rehabilitation services.
It is difficult for me to attach any kind of rank or importance to these items. I feel
that they are all highly important, highly desirable and have the potential for contri-
buting a wide-ranging base of new information to the rehabilitation field. Without a
well-designed research effort constructed to yield useful information to the rehabili-
tation field, I see no point in going deeply into an effort to establish functional
limitations as the basis for the provision of rehabilitation services.
II-A-5-a,b
-- After reviewing my previous comments and others, I see that I made a mistake. These
are clearly direct uses by the researcher, in that scale scores make logical dependent
variables in assessing experimental treatments. These purposes, as well as 3(f) and
3(g) could be served after the scale has been developed for clinical use.
II-A-5-a
-- Change of word to "evaluating" makes this, and b, a more feasible activity.
-- Your new wording is good.
-- "Evaluating" is an improvement. Researchers will have to continually adapt our
measures of limitations to keep pace with technological, social, economic, medical and
environmental changes.
II-A-5-b
-- I think that a functional limitations inventory is more likely to help with evaluation
of new techniques than with development of those techniques.
-- I believe that identification of functional needs, including "market data", sours
development. In this sense, "develop" is a better choice for 5-b than evaluate. Maybe
we are talking about two different tasks, each of which deserves mention.
-
Please write comments on opposite page
)
15.
6. Relevant Others
Importance/
Confidence
Please rate the importance of functional limitation
instruments for the following relevant others in the
rehabilitation process and your confidence in your
response
*a) trainers of rehabilitation personnel -- to assist
rehabilitation personnel: identify more precisely
client needs and matching these needs with available
I
services
b) family members -- to increase supportiveness and to
improve the understanding of the disabled person's
problems
**c) secondary service providers (rehabilitation facili-
ties, training facilities. medical suppliers, etc) --
to assist them in better identifying the needs of
the client
d) other (list and rate)
*This wording has been altered. If you disagree with this
change, please state so and why in your comments
** - Consensus was reached, but respondents may be con-
fused as to the definition of secondary service provider.
It has therefore been defined and re-stated for your
rating
Question
Importance
Comments concerning page 15
Number
4321
II-A-6-b
1431
-c
5211
II-A-6-a
-- Why change the wording so much after the "votes"?
I don't disagree with the new statement, but its
not what I rated before. I think that the former
statement was important also, and not the same as
this one.
II-A-6-b
-- My rating is based on the assumption that the FL scale will be a non-tehcnical and non-
jargon instrument. The high rating of 4 is based on the notion that families need specific
information focused on specific problems and strengths, to enhance their role as social
supports to the client.
II-A-6-c
-- This is a logical extension of clinical use by the counselor.
( Please write comments on opposite page
)
16.
B. Unique Client Populations
All respondents seemed to feel that all the handicapping conditions listed below
must be covered by functional limitations assessment instruments. Furthermore,
all of the identified client populations present unique problems in the develop-
ment of an instrument (or instruments). Those developing functional limitation
instruments should study each disability category to make certain that the
limitations of all disability population groups are included in instruments
which are developed. Failure to do so may result in inequities such as denial
of services to those who need them. The feasibility ratings did vary which
seems to indicate that feasibility is not presently known. Feasibility will
be determined as researchers explore these problems. Finally, many of the
comments were excellent. Most will be included in the final report of this
project.
Mentally Retarded
Blind
Deaf
Deaf-Blind
Hidden Handicaps and Progressive Diseases including:
a) epilepsy
b) diabetes
c) renal disease
d) cancer
e) coronary artery disease
f) pulmonary disease
g) endocrine disorders
Mental Illness
Recently Traumatically Injured
In addition some respondents cited the following additional populations: the
learning disabled, the socially disadvantaged and individuals with chronic
pain, especially low back pain.
Comments concerning page 16
-- Unique Client Populations - Throughout this project it has been my basic assumption
that if the way is cleared for functional limitations to serve as the basis for eligibility
and feasibility that groups of experts in the field will be called together to list and
agree upon to the extent possible the major functional limitations surrounding each dis-
ease entity or disability category that the field of rehabilitation is likely to be dealing
with. I view this as an intensive, long-range procedure that will require the meeting of
minds of a wide variety of people who have expertise in the field of rehabilitation and
ancillary areas. It it is not possible to reach total consensus on what are the functional
limitations for a given disease or disability, then I think some determination should be
made to use the best information available and proceed from there with the assumption that
further additions and deletions can be made as to the scope of functional limitations as
they become known.
Please write comments on opposite page
)
17.
We do feel it important to include Question 9 from previous rounds in a
modified form. Please answer the following question after reading the
responses of the previous Delphi, and giving careful consideration to
the issues involved.
How should instrument developers progress in their work in regard to
number of scales to be developed?
Should try to develop 1 scale
Should try to develop more than 1 scale based on disability
type
For the immediate future, focus on developing more than 1
scale with an eye toward eventual development of 1 com-
prehensive scale
Develop 1 scale with a branching system (sub-scales)
Other (list)
Comments concerning page 17
Question
Number
9
1 Scale
More than 1
1
More than 1, then 1
1 with Subscales
7
II-B-9
-- In my opinion a single summary "score" is necessary to describe client status relative
to a hypothetical "fully functioning person." However, sub-scores are necessary to make
a scale useful for diagnostic and/or prescriptive uses.
-- The final report and meeting should focus in more detail on how "branching" inventories
will work.
-- One scale with branching best fits my concept of a global screening scale to identify
major problems and then subscales to measure more extensively specific types of limitations,
e.g., communication, mobility. This new alternative seems to satisfy most points raised.
-- I have always had some difficulty throughout the entire project with the use of the term
"scale." This was acknowledged by the project designers and the word instrument was in-
serted at certain points in the material sent to us. However, this item continues to revolve
around the use of the term scale. Perhaps I am straining at a gnat here. However, I do
feel this is an important item -- deleting the word "scale" and using the term instrument.
I feel that the ultimate format for expressing information about functional limitations
should be done in a multi-faceted manner with a final expression in a profile that could
be the sum of the significant sub-items of whatever measurement expression instrument is
used. In the final analysis, I think it is really a moot question at this point as to what
the ultimate format should be on the basis of one scale, more than one scale on disability
type, etc. Why not explore the field thoroughly, see what results can be obtained from
attempts to measure and isolate functional limitations, and then determine the best format
for expressing them. This is almost like having a suit of clothes tailor-made at an adult
level for a child that is only 12 years old, when we do not know how much growth is going
to occur or what dimensions that the child may ultimately take.
-- The branching system development of the scale would allow precise coverage without
volumes of papers.
(Please write comments on opposite page
)
18.
C. Criteria for Functional Limitations Instruments
Although numerical consensus was reached on most items in this section,
many concerns were voiced by respondents with regard to:
1) The issues involved here are complex research issues
2) It's too early to answer some questions
3) Some respondents are not "scale-development experts,"
Therefore, the ultimate answers to these questions rest with scale developers
(and the research community, who will accept or reject the adequacy of
the scales). The results of this section will be provided in the final
report as possible directions for future research. A summary of the
numerical results are below. Comments from previous Rounds will be
included as an integral aspect of these numerical ratings.
If you agree with this statement, check this box
If you disagree with this statement, check this
box, and provide any comments you care to make
(Also, feel free to add to, or re-write any
portion of the above statement)
Comments Concerning page 13
Question
Number
II-C
Agree
7
II-C
Disagree - 2
-- It is my impression, from the comments, that the
hesitation is not based on inability to answer
scale-development questions, but that the deter-
mination of criteria for instruments is premature,
pending decisions on previous issues re users and
uses. I would hope, at the proper time in the development process, "scale developers"
seek similar opinions as to criteria (perhaps in another Delphi round). The research
community and the users should continue to be involved in the design process, rather
than merely accept or reject developed scales.
-- These issues are difficult to rate for the reasons cited and in that the design of
FL scales depends on the purposes they must serve and the type of statements they are
meant to produce, e.g., is the FL scale an assessment of what the client does/doesn't
do (in what environment?) or what the client can/can't do (under what test conditions)?
-- Although I agree with the statement, I have strong opinions about many of the items
in this section. In most instances, my opinions correspond to the consensus that was
reached. While it is too early to answer some of the questions, then, I think our
experience already provides the basis for reliable answers to many others.
( Please write comments on opposite page
19.
Summary of numerical ratings
1. Measurement - Scoring Features
Importance/
Confidence
Feasibility/
Confidence
a) nominal, ordinal, equal interval scoring
NI
F
Nominal
-
I
F
Ordinal
*
NF
Equal Interval
I
*
b) overall score of severity
I
F
c) use of subscales
I
F
d) weighting items in relation to one another
2. Reliability (consistency of the scale, obtaining the same
results)
I
F
a) test - retest reliability on a given client
I
F
b) agreement among 2 or more raters
3. Validity (does the item/scale really measure what it
is supposed to measure?)
On scales such as this, true validity measures do not exist,
and one must seek the best available substitute. Below are
Acceptability/
Confidence
Feasibility/
Confidence
some of these substitutes. Please consider each carefully and
rate them.
A
F
a) level of agreement with counselor assessement
b) level of agreement with other functional limitation
*
F
scales
*
*
c) level of agreement with client assessment
*Numerical consensus not reached
Question
Desirability
Feasibility
Comments concerning page 19
Number
43214321
III-A-8
1232 162
III-A-8
-- Am swayed somewhat by majority vote on this. There are large numbers of clients
from whom such a scale would be appropriate. Previous qualifications were mainly
because this area of functional limitation can be attributed to more "basic" limi-
tations of communication, mobility, etc., and therefore measurement may be redundant.
-- This high rating is based on the assumption of a branched FL scale.
-- Both male and female clients should be rated on this dimension, if it is to be
used in any way other than as the normal "vocational" skills covered in (6).
-- Maybe I'm confused by the use of the term skills, but I am not in agreement with
including an assessment of learned skills (such as ability to type) in a scale of
functional limitations. If the person can't type due to problems with muscle control
and/or coordination, then he or she has a lack of skill resulting from a functional
limitation. However, it's obvious that many individuals simply haven't been taught
how to type (This also applies to 4 and 6).
-- Homemaking/child care skills are important not just for individuals working toward
a goal of homemaker. There are key homemaking and child care elements which would not
be assessed for disabled people unless a branching type inventory includes them. They
are important, however, because society and the disabled individual needs to be able
to make judgements on the ability to do these taks. Inability to do them, or conversely,
ability, has definite consequences in terms of such things as child custody decisions,
compensation awards, availability of home care assistants, etc. Insurance companies
for example, seldom recognize the role of the homemaker by selling disability insurance
or disability waiver of premium on life insurance for homemakers. Employee compensation
benefits such as sick leave etc., seldom recognize emergency or short-term disability
related needs of spouses as legitimate use of benefits. All of these situations and
many others are grounds for upgrading and professionalizing of assessment of disability
of homemakers. This is a true example of bias on the part of the males who design and
use assessments in failing to recognize the role of the "home economy. Increasing
recognition of homemaker values in tort cases and in proposed legislation indicates
that this is a rapidly changing area. Arguing that homemaker functions be placed
under another category such as vocational underestimates the specific elements of the
homemaker role. Rather, a branching system should be used to draw out differences
and cross-reference elements of sub-scales.
(Design Team Note: We also received the following comment on this section).
P.S. -- As a male who recently had to face the very difficult challenge of maintaining
my job and maintaining a home with four small children while my wife experienced
severe limitations resulting from pregnancy and thyroid disease, I'm tempted to rate
myself as expert on this point!
( Please write comments on opposite page
20.
Summary of numerical ratings continued
Acceptability/
Confidence
Feasibility/
Confidence
d) prediction of behavior in a wider range of
A
F
settings
e) should be left exclusively to the researcher
*
F
developing the instrument
Importance/
Confidence
Feasibility/
Confidence
4. Brevity/Detail of Scale
*
*
a) brevity
I
F
b) detail
I
F
c) branch system with subscales to be used as needed
5. Objectivity-Subjectivity of items
I
F
a) objectivity of items
*
*
b) subjectivity of items
*
*
6. Predictiveness
I
F
7. Descriptiveness
I
F
8. Prescriptiveness
I
F
9. Ease of Interpretation
I
F
10. Ease of Training in Usage
*Numerical consensus not reached
Please
write
comments
on
21.
III. COMPONENTS OF FUNCTIONAL LIMITATIONS INSTRUMENTS
A. Strict Functional Limitations Items
Consensus was reached on the following components of a functional limitations
instrument, except for number 8, homemaking. Each of the others were seen
as both desirable and feasible. Please rate number 8 again.
Desirability/
Confidence
Feasibility
Confidence
1. Mobility (Manual skills; locomotion short/long
D
F
distances)
-
-
2. Communication (sending/receiving information to/
D
F
from other persons)
-
-
3. Cognitive-Intellectual (learning; manipulation of
symbols and objects; transference of learning to new
D
F
situations
D
F
4. Self-Care Skills (bathing, grooming, eating, etc.)
-
-
D
F
5. Emotional (situational appropriateness)
D
F
6. Vocational Skills
-
7. Social Competence (interactions with other
D
F
people)
-
-
8. Homemaking/Child care
B. Other Direct Supportive Information
Consensus was obtained on most items in this section, but it was pointed out
that these are not functional limitations. While these items are not functional
limitations, they are items which may add to the understanding of functioning,
and may provide a valuable adjunct to functional limitation instruments. They
will therefore be presented in the final report, but it will be noted that they
are important supportive information only (not functional limitations).
Please write comments on opposite page
22.
Below are listed the direct supportive information items previously rated and agreed
to as important and feasible. Please rate numbers 2 and 5 again as only partial
agreement on those items had been reached on Round 2.
Importance/
Confidence
Feasibility/
Confidence
I
F
1. Exceptional Positive Personal Characteristics
2. Exceptional Negative Personal Characteristics
I
F
3. Health Problems
I
F
4. Vocational History
1.
5. Social - Attitudinal
I
F
6. Demographic Information
Consensus was reached on the issue of including the client's ability to utilize or
cope with environmental factors, however, including the same environmental factors
without reference to the client was rejected.
7. Environmental items which measure the client's ability to utilize or cope
with environmental factors to his advantage.
I
F
i) attitudes of others
ii) presence of technology relevant to the handi-
I
F
capped person's needs
-
I
F
iii) availability of services
iv) physical accessibility of home, work and
I
F
community
I
F
v) transportation
Importance
Feasibility
Comments concerning page 22
Question
Number
4
3
2
1
4
3
2
1
III-B-2
141125
III-8-5
16
1
4
2
III-8-5
-- I am willing to concede that this is an important area. Again, however, we need
to strive for items that are as behavioral and objective as possible.
-- Going along with a comment in the Round 2 results, I agree that behavioral mani-
festations of these variables are preferable and more feasibly developed. I would
rate these variables 4, if it were clear to me that social behaviors were being
referred to.
( Please write comments on opposite Dade
23.
IV. UTILIZATION
A. Facilitation of Utilization
Consensus was reached on most items listed below. Generally, the approaches listed
were seen as effective, feasible and important. One item still needs to be rated,
number 7, a and b.
Probable
Effectiveness*/
Confidence
Feasibility/
Confidence
Importance/
Confidence
1. Incorporating the use of functional limitation scales
into the training programs of rehabilitation professionals
E
F
I
(including in-service training)
-
I
-
2. Design of technical assistance teams to assist State
VR agencies in implementing the functional limitation
E
F
I
concept
-
-
3. Design of technical assistance teams to assist
facilities and other programs in using functional
E
F
I
limitation scales
1
4. Design of seminars/conferences to relay findings
E
F
I
to researchers, administrators and potential users
E
F
I
5. Funding of demonstration projects
-
6. Lobbying for and. supporting legislation focusing
efforts on the conceptualization of disability based on
E
I-1
functional limitations
7. If one of the main user groups of such an instrument(s) is the counselor,
he will be using the Functional Limitation results to develop the IWRP with
the client. The client might therefore want to be aware of how he could
utilize functional limitations information. What are the most effective ways
to facilitate the client's use of functional limitation instrument information?
a) Design of seminars/conferences to introduce
clients to functional limitations concepts and
instruments
b) Training counselors to work with clients on a one-
to-one basis, to interpret functional limitations
information and explain its use
*Use the definition of "desirability" in the front of this
Questionnaire for the definition of probable effectiveness
Comments concerning page 23
Probable
Question
Effectiveness
Feasibility
Importance
Number
4
3
2
1
4
3
2
1
4
3
2
1
IV-A-6
1
-7-a
153
8
1
153
-b
261
261
351
IV-A-7
-- Participant responses to this question seem consistent with their views on
direct usefulness of the scale to client. If scale is to be used by the client,
training counselors to interpret the information seems reasonable. I have upgraded
my response to 7(a) after reconsidering the modest value of client group orientation
to the rehab process.
-- Large-scale conferences to train clients is probably infeasible, particularly for
severely disabled clients; one possible approach would be a training videotape and
information packet, as part of a "client rights" introductory package that every
client should see prior to the IWRP process. The idea that the counselor should
interpret the scale to the client seems inappropriate. Some separate client advocate
outside or inside the system should play that role.
-- Both of these methods would be desirable and effective. However, I strongly favor
either as a component of a and b, or as a new option "c", "peer counseling". I would
have disabled individuals themselves, or in some cases, parents or advocates, train/
orient other clients in use of functional inventories.
IV-A-7-a,b
-- Your new wording shows me that I misinterpreted a and b on the last round. I have
many doubts about direct use of functional limitations inventories by clients. If
they do turn out to be useful at all, it seems likely that it will be in the counseling
interview with the counselor serving as interpreter.
I
Please write comments on opposite page
)
24,
IV. UTILIZATION
A. Facilitation of Utilization (continued)
Probable
Effectiveness*/
Confidence
Feasibility/
Confidence
Importance/
Confidence
8. Please rate the effectiveness of National Organizations
advocating the use of functional limitations concepts, the
feasibility of encouraging such organizations to do so,
E
I
and the importance of encouraging them
-
9. Having potential users assist in the development of
E
F
I
the instrument
*Use the definition of "desirability" in the front
of this Questionnaire for the definition of probable
effectiveness
Comments concerning page 24
Question
Feasibility
Number
4321
IV-A-8
131
2111M
saue
on
opposite
Dade
B. Barriers to Utilization
25.
A mistake was made in the last round. We intended to get a rating of likelihood
and importance (to some extent, this was asked for in a combined rating -- see
last round directions on page 24). The responses received could not be
interpreted, so we are asking you to rate 1) the likelihood (probability) of each
item occurring, and 2) Importance of each barrier, if it does occur (i.e., is
it a significant barrier which will require substantial resources to overcome
if it does occur).
Likelihood/
Confidence
Importance/
Confidence
1. Counselor Concerns
a) fear of increased work load
b) fear of accountability
i) to supervisor and to other agency staff
ii) to client
c) suspicion about any new technique
d) fear that any failures or inadequacies will be
discovered and exposed
e) possible lack of proper training in utilization
of scales
f) fear of depersonalization of the counseling process
g) other (list and rate)
2. Possible Non-acceptance by Consumer Groups
a) fear of loss of a benefit
b) fear of further categorization
c) fear of over-emphasis on functional limitations
d) fear that scale will not be relevant to their
disability
e) disability based lobbies may fight change
f) client may fight change because of expenditure of
time and effort in evaluation process
g) other (list and rate)
Likelihood
Comments concerning page 25
Question
moortance
Number
4
3
2
1
4
3
2
1
IV-B-1-a
2
4
3
3
2
4
-b
2
1
3
1
1
4
-b-i
2
1
5
1
3
4
-ii
1
2
5
3
4
1
-c
2
2
5
1
3
5
-d
1
8
2
2
5
-e
4
4
3
5
-f
3
6
2
1
6
IV-B-2-a
3
1
4
3
3
2
-b
2
3
4
2
4
3
-c
2
4
3
1
5
3
-d
3
1
5
2
3
4
-e
3
4
1
4
5
-f
1
7
1
i
7
IV-8
-- I see importance and likelihood as closely related in practice with 2 or 3 minor
exceptions.
Please write comments on opposite page
)
26.
Likelihood/
Confidence
Importance/
Confidence
3. State Agency Concerns
a) fear of increased reporting requirements
b) fear of exposure of State agency failures or
inadequacies
c) lack of training resources
d) change in bureaucracy takes time and staff
e) cost of change
i - organizational
ii - because of increase in eligible population
f) fear of increased control by Federal government
*g) different Federal programs have different eligibil-
ity requirements. The functional limitations
scale(s) selected may not provide enough information
to determine eligibility for the VR program or may
not be able to determine eligibility for other
Federal/State programs (e.g., similar benefits
such as Medicaid, Medicare, Title II, etc.)
h) other (list and rate)
4. RSA Concerns
a) cost of change
i - organizational
ii - because of an increase in the ineligible
population
b) change in a bureaucracy takes time and staff
*This question was dropped as it is felt that many issues were
raised in one question, and the important issues were addressed
elsewhere
Likelihood
page
Question
Importance
Number
43214321
IV-B-3-a
3
4
2
4
4
1
-b
1
1
7
5
4
-c
4
3
2
5
7
2
-d
2
5
2
1
4
4
-e-i
3
4
2
2
4
3
-ii
4
4
2
3
3
-f
1
1
7
1
1
7
4-a-i
1
5
2
2
4
2
-ii -
3
5
2
2
4
-b
1
6
1
1
4
3
IV-B-4-a-ii
-- eligible not ineligible
Please write comments on opposite pade
27.
4. RSA Concerns (continued)
Likelihood/
Confidence
Importance/
Confidence
c) the functional limitation scale(s) chosen may con-
flict with RSA's Congressionally-mandated defini-
tion of handicap or the definitions used by other
Agencies
d) Applying the functional limitation scales into
the eligibility process
e) fear that without Federal legislation, functional
limitations instruments will be adopted by only
some States, and in different ways, which may lead
to disparities in the VR program
f) other (list and rate)
Likelihood
Importance
Comments concerning page 27
Question
Number
43214321
IV-B-4-c
134
143
-d
332
233
-e35
341
(thease write comments on opposite page
28.
C. Potential Abuses
In this Section, we obtained consensus on most items. On the remaining items,
we are seeking opinion on potential abuses which are likely to occur. Please
rate the importance of the following items as potential abuses of functional
limitations assessment instruments (e.g., Is each abuse an important one with
which we should be concerned? To what degree?).
Importance/
Confidence
1. Denial of Services to clients who actually
should be receiving them
2. "Pigeonholing" of Clients into severity categories
a) functional limitations scales may lead to labeling
which could have negative consequences for the
client
b) reliance on functional limitation scales without
use of other relevant information may result in
I
inappropriate service delivery
3. Administration of Function Limitations Scales Without
Proper Training
I
a) by counselor
I
b) by support staff
c) others ( persons not in the counseling situation,
e.g., employers, family members, etc.)
4. Using (interpreting) Results of Functional Limitations
Scales without proper training/understanding
I
a) by counselor
I
b) by support staff
c) by others
5. "Fudging" by Counselors
Comments concerning page 23
Question
Importance
Number
4
3
2
1
IV-C-1
6
3
-2-a
2
5
2
IV-C-3,4
-3-c
2
4
1
-4-c
2 4 2
-- Specific attention should be given to the role
-5
1
5
3
of peer counselors and other consumers (parent
groups, consumer advisory groups) as to their
needs for training and orientation to FL inventories.
IV-C-1
-- My reason for rating this as a potential danger of only some importance, was due to the
probability that, when a new system of eligibility determination is introduced, there will
be inequities for a while until errors of measurement and judgement are reduced. I cannot
understand the "very important" rating by other participants. If a client has no measur-
able functional limitations, I cannot see why that client could otherwise be considered
eligible for service.
IV-C-2-a
-- Possible abuse here can be avoided with some care.
IV-C-2-c
-- Abuse can be avoided by agency administrative regulations concerning administration
procedures.
29.
Importance/
Confidence
6. Inappropriate Use of Scales
I
a) use of scale prior to its final development
b) use with population other than the one it was
I
designed for
D. Other Uses of Functional Limitations Scales
All items were seen as highly desirable and feasible.
1. Cost-benefit/effectiveness studies
2. Development of National data base
3. Use by other service programs to determine eligibility