Computerized Adaptive Testing System for Measuring Self-Care Performance

Instrument Name: Computerized Adaptive Testing System for Measuring Self-Care Performance

Abbreviation: CAT-SC

Points for Consideration:

New tool that has done some good work in development. Limited research available.

Description of Tool:

The Computerized Adaptive Test for Gross-Motor Skills is an ObsRO which includes up to 56 items on the child's ability to complete self-care activities in children aged 6 months to 12 years. Activities range from indicating the need for food or drink, ability to wash hands, dress, hygiene, and urinary and defecation regulation. Activities are rated as requiring "total assistance, " "partial assistance," or "No assistance" and use a Rasch model to derive a total score. Higher scores represent better self-care performance.

Minimum Qualification Required by COA Administrator: No degree requirement

Year: 2018

Objective of Development:

To measure self-care performance in children with DD (developmental disabilities) whose performances are comparable to those of TD children aged from 6 months to 12 years

Population of Development: Age range (therapeutic indication):

6 months to 12 years (Developmental Disabilities)

Pediatric Population(s) in which COA has been used:

Mental disorders; Nervous system diseases; Congenital, Hereditary, and Neonatal Diseases and Abnormalities

COA type:

Number of Items 56

Mode of Administration:

Data Collection Mode:

Time for Completion: 5 minutes

Response Scales: 4-point verbal rating scale from 0 "Total assistance" to 4 "No assistance"

Summary of Scoring:

Available Scores:
Global score (Rasch transformed score were not reported)

Weighting:
No

Score Interpretation:
Higher score = Better gross motor skills

Evidence of Literature Review: Yes

Evidence of Instrument Review: Yes

Evidence of Clinical or Expert Input: Yes

Evidence of concept elicitation in target patient population: None identified

Evidence of a Saturation Grid: None identified

Evidence for Selection of Data Collection Method: None identified

Recall/Observation Period:

Evidence for Selection of Reponse Options: None identified

Evidence of cognitive interviewing of draft instrument in target patient population: Yes

Evidence of Preliminary Scoring of Items and Domains: Yes

Evidence related to respondent and administrator burden: None identified

Evidence of a Conceptual Framework: None identified

Evidence of an item-tracking matrix: None identified

Evidence related to item selection: Yes

Evidence of re-testing the final version: None identified

Internal consistency (Cronbach's alpha): None identified

Test-retest Reliability (ICC):

Huang CY (2020)

- Intraclass Correlation Coefficient (ICC): 0.99
- Was a definition of stability applied to identify stable patients: No
- Time frame or interval between the two administrations: 1 week
- Population/Disease: Children with developmental disabilities; n=30, from a sample of 60 children, mean age= 5.21±2.16 years

Inter-rater/ inter-interviewer reliability (kappa):

Huang CY (2020)

- Intraclass Correlation Coefficient (ICC): 0.92
- Population/Disease: Children with developmental disabilities; n=30, from a sample of 60 children, mean age= 5.21±2.16 years

Evidence of test-retest or inter-rater reliability: Yes

Concurrent validity (convergent, divergent):

Huang CY (2020)

- Correlation coefficient used: Pearson's correlation coefficient
- Measure: Pediatric Evaluation of Disability Inventory Chinese version (PEDI)
- Results: Significant correlations were found between CAT-SC and both PEDI domains: p<0.001 Functional skills scale: 0.92 Caregiver assistance scale: 0.91 - Population/Disease: Children with developmental disabilities; n= 60, mean age= 5.21±2.16 years

Known-group validity:

None identified

Evidence of Translatability Assessment: None identified

Evidence related to missing data: None identified

Evidence for Selection of Recall Period: Yes

Evidence of Administration Instructions and Training Provided: None identified

Evidence of concurrent validity: Yes

Evidence of known-groups validity: None identified

Evidence of ability to detect change over time: Yes

Ability to detect change (Responsiveness):

Huang CY (2020)
Methods: Distribution-based
- Population/Disease: Children with developmental disabilities; n(Baseline)=60, n(3 months= 56) and n(6 months)=53, mean age= 5.21±2.16 years for the n(Baseline)
- Time horizon: 3 and 6 months
- Statistics used: Standard Response Mean (SRM), Cohen's d effect size (ES)
Results: Significant difference in scores was found between baseline and 3 months and baseline and 6 months: p<0.05 Small effect size was found between baseline and 3 months: ES=0.02, SRM: 0.08 Small effect size was found between baseline and 6 months: ES=0.12, SRM: 0.33

Responder Thresholds:

Huang CY (2020)
- Population/Disease: Children (n3 months= 17 and n6 months= 9) with developmental disabilities with small changed as rated with a 15-point Likert Scale by Caregivers, age: stgated
- Method used: Anchor-based
Caregviers rated children with a 15-point Likert type scales:
Between baseline and 3 months: 17 children were rated as having small changes with the scale.Among them 1 children had declined, 2 had unchanged performance and the other children had improved. Mean change score of the 2 assessments was assessed
Between baseline and 6 months: 9 children were rated as having small changed the scale. Among them, 1 child had declined performance, 1 had unchanged and the other had improved. Mean change score of the 2 assessments was assessed
- Results:
CID between baseline and 3 months: 0.17 logits
CID between baseline and 6 months: 0.10 logits

Evidence of responder thresholds: Yes

Chen CT, Chen YL, Lin YC, Hsieh CL, Tzeng JY, Chen KL. Item-saving assessment of self-care performance in children with developmental disabilities: A prospective caregiver-report computerized adaptive test. PLoS One. 2018 Mar 21;13(3):e0193936
(Full text article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5862472/pdf/pone.0193936.pdf)

Huang CY, Chen SS, Chen CT, Lee PS, Yu TY, Chen KL. Psychometric Properties and Efficiency of the Computerized Adaptive Testing System for Measuring Self-Care Performance in Taiwanese Children With Developmental Disabilities. Arch Phys Med Rehabil. 2020 Aug;101(8):1332-1337 (PubMed abstract: https://pubmed.ncbi.nlm.nih.gov/32109435/)

See (PubMed results)
https://pubmed.ncbi.nlm.nih.gov/?term=%22Daily+Activities+of+Infants+Scale%22&sort=date

None identified

Original language: Chinese for Taiwan

Translation:
English

None identified

Prof. Cheng-Te Chen
Department of Educational Psychology and Counseling
National Tsing Hua University
Hsinchu
Taiwan
Republic of China
E-mail: chengte@mx.nthu.edu.tw

Not reported

Not reported

Copy of the English translation can be found in Chen et al. 2018 (S1 Table) (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5862472/pdf/pone.0193936.pdf)