Cancer Control
P.L.A.N.E.T. Evaluation
Final Report

Authors:

Rosie Sood
Pei-Shu Ho
Carina Tornow
William Frey



The findings and conclusions in this report are those of the
authors and do not necessarily represent the views of the
funding agency.



November 2007

Prepared for:
Prepared by:


Division of Cancer Control and
WESTAT
Population Sciences
Rockville, Maryland
National Cancer Institute

Bethesda, Maryland

---

---

TABLE OF CONTENTS
Chapter
Page

EXECUTIVE
SUMMARY .............................................................................
xi

ACRONYM
LIST............................................................................................
xvii

1 INTRODUCTION ...........................................................................................
1-1

1.1 Background......................................................................................... 1-2
1.2
Development and Evolution of Cancer Control P.L.A.N.E.T. ...........
1-5

2 METHODOLOGY ..........................................................................................
2-1

2.1 Web
Usage
Data ................................................................................. 2-1
2.2
Assessment of Intermediate Outcomes (AIO) ....................................
2-3

2.2.1 AIO
Development ............................................................... 2-3
2.2.2
AIO Sample Selection and Implementation........................
2-5
2.2.3
AIO Respondent Characteristics .........................................
2-7

2.3
Usability and Customer Satisfaction (UCS) .......................................
2-11

2.3.1 UCS
Development .............................................................. 2-11
2.3.2
UCS Sample Selection and Implementation .......................
2-12
2.3.3
UCS Respondent Characteristics ........................................
2-14

2.4
Research-Tested Intervention Programs (RTIPs) Requests Data .......
2-17

2.4.1 RTIPs
Development ............................................................ 2-18
2.4.2
NCI Warehouse RTIPs and Developer RTIPs ....................
2-19

3 RESULTS ........................................................................................................
3-1

3.1
Web Usage Results .............................................................................
3-1

3.1.1
Cancer Control P.L.A.N.E.T. Web Site Usage Results.......
3-2
3.1.2
RTIPs Web Site Usage Results ...........................................
3-3

3.2 AIO
Results ........................................................................................ 3-9

3.2.1
Factors Pertaining to AIO Respondents’ Utilization of
Cancer Control P.L.A.N.E.T. ..............................................
3-10
3.2.2 State
Cancer
Profiles ........................................................... 3-13
3.2.3
Cancer Control Partners ......................................................
3-14
3.2.4 Evidence
Reviews ............................................................... 3-15
iii

---

TABLE OF CONTENTS (continued)
Chapter
Page

3.2.5
Research-Tested Intervention Programs (RTIPs) ...............
3-16
3.2.6
Planning, Implementing, and Evaluating Comprehensive
Cancer Control Programs ....................................................
3-20

3.3 UCS
Results........................................................................................ 3-22

3.3.1
Factors Pertaining to UCS Respondents’ Utilization of
Cancer Control P.L.A.N.E.T. ..............................................
3-23
3.3.2
Cancer Control P.L.A.N.E.T. Information Sought and
Used by UCS Respondents .................................................
3-27
3.3.3
UCS Respondents’ Experience Using Cancer Control
P.L.A.N.E.T. ....................................................................... 3-33

3.4 RTIPs
Requests Results ......................................................................
3-37

4
DISCUSSION AND SUMMARY...................................................................
4-1

5 RECOMMENDATIONS.................................................................................
5-1

5.1
Strategies for Meeting the Goals of Cancer Control P.L.A.N.E.T. ....
5-1
5.2
Strategies for Developing a Future Long-Term Outcome Evaluation
5-2

REFERENCES ................................................................................................
R-1


List of Appendixes

Appendix

A
Cancer Control P.L.A.N.E.T. Trainings ..........................................................
A-1

B
Cancer Control P.L.A.N.E.T. Exhibits ............................................................
B-1

C
P.L.A.N.E.T. Assessment of Intermediate Outcomes (AIO) Survey...............
C-1

D
AIO Letter from NCI .......................................................................................
D-1

E
AIO Survey Pilot Version 1.............................................................................
E-1

F
AIO Survey Pilot Version 1 Report .................................................................
F-1

G
AIO Survey Pilot Version 2.............................................................................
G-1

iv

---

TABLE OF CONTENTS (continued)
List of Appendixes (continued)

Appendix

H
AIO Survey Pilot Version 2 Report .................................................................
H-1

I
AIO Cognitive and Usability Testing Email....................................................
I-1

J
AIO Cognitive and Usability Testing Debriefing ............................................
J-1

K
AIO Usability Testing Report..........................................................................
K-1

L
AIO Invitation Email .......................................................................................
L-1

M
AIO Reminder 1 Email ....................................................................................
M-1

N
AIO Reminder 2 Email ....................................................................................
N-1

O
AIO Cover Letter Error Group ........................................................................
O-1

P
AIO Audio File Script from Jon Kerner ..........................................................
P-1

Q
AIO Audio File Email......................................................................................
Q-1

R
Corrected AIO Audio File Email .....................................................................
R-1

S
AIO Respondents Age Group by Occupation and Work Setting.....................
S-1

T
Cancer Control P.L.A.N.E.T. Usability Satisfaction Survey ...........................
T-1

U
UCS Cognitive Testing Email .........................................................................
U-1

V
UCS Cognitive Testing Version 1 ...................................................................
V-1

W
UCS Cognitive Testing Report Round 1 .........................................................
W-1

X
UCS Cognitive Testing Version 2 ...................................................................
X-1

Y
UCS Cognitive Testing Report Round 2 .........................................................
Y-1

Z
Email Blasts Message Text..............................................................................
Z-1

AA
Evaluation Survey Fact Sheet ..........................................................................
AA-1

BB
Cancer Control P.L.A.N.E.T. Listserv Message about Surveys ......................
BB-1
v

---

TABLE OF CONTENTS (continued)
List of Appendixes (continued)

Appendix

CC
Audio File Script for Cancer Control P.L.A.N.E.T. Listserv...........................
CC-1

DD
UCS Respondents Age Group by Occupation and Work Setting ....................
DD-1

EE
Research Tested Intervention Programs (RTIPs) – Posting Dates

and Topic Areas ...............................................................................................
EE-1

FF
RTIPs Posting Dates Breakdown.....................................................................
FF-1

GG
RTIPs Program Review Ratings ......................................................................
GG-1

HH
Elements of Research Integrity ........................................................................
HH-1

II
Monthly Average Program Summary Views by Topic Areas Breakdown......
II-1

JJ
Monthly Average Product Previews by Topic Areas Breakdown ...................
JJ-1

KK
Number of Users Redirected to Developer Web Site by Topic Areas
Breakdown .......................................................................................................
KK-1

LL
AIO Qualitative Data .......................................................................................
LL-1

MM
AIO Respondent Number of P.L.A.N.E.T. Visits by Occupation and
Work
Setting ....................................................................................................
MM-1

NN
AIO Respondents’ Settings and Occupation by Ways RTIPs Were Used

for Each Program Type....................................................................................
NN-1

OO
UCS Qualitative Data ......................................................................................
OO-1

PP
UCS Respondent Number of P.L.A.N.E.T. Visits by Occupation and
Work
Setting ....................................................................................................
PP-1

QQ
UCS Respondent Ratings on Revisiting P.L.A.N.E.T. by Occupation and
Work
Setting ....................................................................................................
QQ-1

RR
NCI RTIPs Requests by Topic Areas ..............................................................
RR-1

SS
Definitions and Examples of Customer Categories .........................................
SS-1

TT
NCI RTIPs Customer Type by Topic Areas .................................................... TT-1
vi

---

TABLE OF CONTENTS (continued)
List of Tables

Table
Page

2-1
Cancer Control P.L.A.N.E.T. evaluation data sources ....................................
2-1

2-2
Monthly Web usage trends reports - variables list...........................................
2-2

2-3
AWStats Web usage data - variables list .........................................................
2-2

2-4 AIO
mailings....................................................................................................
2-6

2-5
Demographic characteristics of AIO respondents ...........................................
2-8

2-6
AIO respondents’ work settings by occupations (n = 140)..............................
2-9

2-7
Number of AIO respondents by state...............................................................
2-10

2-8
Region of AIO respondents’ work settings (n = 134)......................................
2-11

2-9
Demographic characteristics of UCS respondents...........................................
2-14

2-10
UCS respondents’ work settings by occupations (n = 133) .............................
2-15

2-11
Number of UCS respondents by State .............................................................
2-16

2-12
Region of UCS respondents’ work settings (n = 130) .....................................
2-17

3-1
Distribution of duration of Web site visits (average number of visits
per
month)........................................................................................................
3-3

3-2
Number of RTIPs Posted (P) and Average Number of Monthly

Downloads (D) by Topic Area.........................................................................
3-8

3-3
Internet use and ways of finding out about Cancer Control P.L.A.N.E.T. ......
3-10

3-4
Ways of finding out about resources on Cancer Control P.L.A.N.E.T.

by respondent’s occupation .............................................................................
3-11

3-5
AIO respondents’ frequency of Cancer Control P.L.A.N.E.T. visits...............
3-12

3-6
Number of times using information on Cancer Control P.L.A.N.E.T. ............
3-12

3-7
Experience using data provided in Step 1 of Cancer Control

P.L.A.N.E.T. (n = 125) ....................................................................................
3-13
vii

---

TABLE OF CONTENTS (continued)
List of Tables (continued)

Table
Page

3-8
Usefulness of data provided in Step 1 of Cancer Control P.L.A.N.E.T...........
3-14

3-9 Potential
partners .............................................................................................
3-14

3-10
Type of Cancer Control P.L.A.N.E.T. resources used for work ......................
3-16

3-11
Experience using the Research-Tested Intervention Programs (RTIPs)..........
3-17

3-12
Ease and method of access to RTIPs programs ...............................................
3-18

3-13
Ways RTIPs were used by RTIPs program types (n = 79) ..............................
3-19

3-14
Tools on Step 5 of Cancer Control P.L.A.N.E.T. and cancer control

program planning involvement (n = 135) ........................................................
3-21

3-15
Internet use and ways of finding out about Cancer Control P.L.A.N.E.T. ......
3-23

3-16
Ways of finding out about resources on Cancer Control P.L.A.N.E.T.

by respondents’ occupations ............................................................................
3-24

3-17
UCS respondents’ frequency of Cancer Control P.L.A.N.E.T. .......................

visits (n = 133) .................................................................................................
3-26

3-18
Reasons for visiting Cancer Control P.L.A.N.E.T. (n = 137) ..........................
3-27

3-19
Reasons for visiting Cancer Control P.L.A.N.E.T. by respondents’
occupations ......................................................................................................
3-28

3-20
Topics of information or resources sought on Cancer Control

P.L.A.N.E.T. (n = 137) ....................................................................................
3-29

3-21
Topics of information sought on Cancer Control P.L.A.N.E.T.

by respondents’ occupations ............................................................................
3-30

3-22
Ways of using obtained information (n = 137) ................................................
3-31

3-23
Ways of using obtained information or resources by respondents’
occupations ......................................................................................................
3-32

viii

---

TABLE OF CONTENTS (continued)
List of Tables (continued)

Table
Page

3-24
Amount of wanted information found on Cancer Control

P.L.A.N.E.T. (n = 130) ....................................................................................
3-33

3-25
Level of usefulness of Cancer Control P.L.A.N.E.T. components ..................
3-34

3-26
Respondents’ ratings on purpose and relevancy of Cancer Control
P.L.A.N.E.T.
...................................................................................................
3-35

3-27
Respondents’ ratings on ease of use of Cancer Control P.L.A.N.E.T. ............
3-36

3-28
Percentage of requests by customer types and topic areas...............................
3-40


List of Figures

Figure

2-1
Number of AIO respondents by day ................................................................
2-7

2-2
States of AIO survey respondents....................................................................
2-10

2-3
UCS respondents by day..................................................................................
2-13

2-4
States of UCS survey respondents ...................................................................
2-16

2-5
Cumulative number of RTIPs in each topic area by month .............................
2-18

2-6
Number of RTIP programs by topic area and distribution source ...................
2-20

3-1
AWSTAT CCP Web usage statistics ...............................................................
3-2

3-2
Average number of views per product or program ..........................................
3-4

3-3
Monthly average program summary views by topic areas ..............................
3-5

3-4
Monthly average product previews by topic areas...........................................
3-6

3-5
Number of users redirected to developer Web site by topic ............................
3-7

ix

---

TABLE OF CONTENTS (continued)
List of Figures (continued)

Figure
Page

3-6
Overview of AIO survey data collection .........................................................
3-9

3-7
Ways of finding out about resources on Cancer Control P.L.A.N.E.T by
respondent’s
occupation ..................................................................................
3-25

3-8
NCI warehouse requests monthly and cumulative...........................................
3-37

3-9 Cumulative
requests
by topic area ...................................................................
3-38

3-10
Total requests by customer type ......................................................................
3-39


x

---

Document Outline

• ÿ
• ÿ
  • ÿ
• ÿ
  • ÿ
• ÿ
  • ÿ
• • ÿ
• ÿ
  • ÿ
  • ÿ
  • ÿ
• • ÿ
  • ÿ
• • ÿ
  • ÿ
  • ÿ
  • ÿ
• ÿ
• Appendixes 11-26-07.pdf
  • ÿ
  • • ÿ
• Posting Date

---