Management and Operations of Online Programs

Contributed by:

Sharp Tutor Fri, Jan 07, 2022 05:52 PM UTC

Online learning is growing rapidly as states and districts are creating new online schools, and existing
programs are adding new courses and students. The growth reflects the spreading understanding
that online courses and programs can serve a wide variety of students and needs.
We will talk about them in this pdf.

1. April 2009
Promising Practices
in online learning
Management and Operations
of Online Programs

2. PROMISING PRACTICES
IN ONLINE LEARNING
Management and Operations
of Online Programs: Ensuring
Quality and Accountability
Written by
John Watson and Butch Gemin
Evergreen Consulting Associates
April 2009
The Promising Practices series is supported by:
®

3. About Promising Practices in Online Learning
Online learning within K-12 education is increasing access and equity by making high quality
courses and highly qualified teachers available to students. Online learning programs offer courses,
academic credits and support toward a diploma. They vary in structure, and may be managed by a
state, district, university, charter school, not-for-profit, for-profit, or other institution. Thirty states
and more than half of the school districts in the United States offer online courses and services,
and online learning is growing rapidly, at 30% annually. This growth is meeting demand among
students, as more than 40% of high school and middle school students have expressed interest in
taking an online course.
The most well established K-12 online learning programs are more than ten years old, and many
programs have between five and ten years of operating experience. The newest programs are
building on the expertise of those early adopters, as well as the experience of online learning in
postsecondary institutions and the corporate world. A body of knowledge, skills and practices
has been developed by individual programs, in collaboration with practitioners, researchers, and
policymakers. Because there are so many types of online programs (full-time, supplemental, state-
led, district-level, consortium), there are also many different approaches to teaching, student
support, professional development, and other issues.
This series, Promising Practices in Online Learning, explores some of the approaches being taken
by practitioners and policymakers in response to key issues in online learning in six papers being
released throughout 2008 and 2009:
Blended Learning: The Convergence of Online and Face-To-Face Education
Using Online Learning for Credit Recovery and At-Risk Students
Management and Operations of Online Programs: Ensuring Quality and Accountability
Socialization in Online Programs
Funding and Legislation for Online Education
A Parents’ Guide to Choosing the Right Online Program
The title, Promising Practices, deliberately avoids the term “best practices.” There are too many
approaches to online learning, and too many innovative teaching and learning strategies in the
21st century, for one method to be labeled “best.” Instead, this series aims to discuss the issues
and explore examples from some of the many online programs across the country, with a goal of
illuminating some of the methods showing the most promise.
Online learning offers the advantage of personalization, allowing individualized attention and
support when students need it most. It provides the very best educational opportunities to all
students, regardless of their zip code, with highly qualified teachers delivering instruction using the
Internet and a vast array of digital resources and content. Through this series of white papers, we
are pleased to share the promising practices in K-12 online learning that are already under way.

4. Management and Operations of Online
Programs: Ensuring Quality and Accountability
Online learning is growing rapidly as states and districts are creating new online schools, and existing
programs are adding new courses and students. The growth reflects the spreading understanding
that online courses and programs can serve a wide variety of students and needs. These include:
Creating opportunities for small and rural school districts to offer varied course subjects and
highly qualified teachers to their students
Allowing students to blend high school and post-secondary learning options
Reducing class size
Helping students recover credits in an alternative learning environment
Providing individualized instruction and unique learning options
Allowing students the opportunity to interact with students far beyond their school or town
boundaries
Meeting the needs and expectations of today’s millennial students1
Many school leaders are excited about the possibilities of online learning. When they start an online
school, however, they quickly confront all the challenges of managing a high-quality, successful
online program: creating online courses; finding, hiring, and managing teachers; supporting
students; managing technology; and evaluating their programs to determine if they are successful.
Fortunately, many online schools have years of operating experience, have developed and revised
formal operations and management structures, and provide examples of successful management.
This paper explores emerging practices in online program management and operations that can be
used by many people working with an online learning program, from executive-level school leaders
to department managers to teachers trying to find ways to improve their effectiveness with online
students. Although it does not address state or district policy issues, legislators and policymakers
will find it useful to understand the varied approaches that online schools are embracing to ensure
quality as they determine the best ways to create oversight while allowing innovation to meet the
needs of students and schools.
A longer list of the opportunities that online learning can provide, from which the above list is taken, has been developed by
consultant David Glick and is provided in Appendix A.
PROMISING PRACTICES 3

5. The first step: determining online program type and goals
The practices that online programs put into place vary according to their program type and
goals. Two of the basic parameters are whether the program is full-time or supplemental, and its
geographic reach. The full-time/supplemental parameter distinguishes whether the online program
is responsible for students’ state assessment scores, Adequate Yearly Progress, and other measures
that are common to all public schools. The geographic reach determines whether the program
has to operate fully at a distance or whether it can easily have a face-to-face component, and also
determines the jurisdictional issues that a program faces (for example a multi-state program has to
address content standards and teacher licensure across numerous states). Many management issues
are dependent on one or both of these two parameters.
In addition to the top-level program characteristics listed above, there are many other attributes
that each program must determine for itself as it considers how best to manage its operations,
including the types of students, whether teachers will be full-time or part-time, and whether
curriculum will be licensed, home-grown or both. Determining these factors is a key component for
a new program that is in its early planning stages, but understanding them can be useful as well
for existing programs that are undergoing strategic planning and may ask themselves, “What are
our characteristics now, and what do we want them to be in the future?” Figure 1 provides a list of
attributes for consideration.
Many of these elements operate along a continuum instead of being an either/or proposition. In
addition, as programs grow and evolve over time, scalability is one of the major challenges that
they face. Growth impacts every aspect of the organization, placing even greater importance on
oversight and management.
With the above characteristics in mind, the following sections review promising practices in the
management and operations of online learning programs in the following areas:
Curriculum development and course quality;
Teacher management;
Student support;
Technology; and
Program evaluation.
For each topic, key issues and variables are identified, and examples of successful management
are explored.
4 Management and Operations of Online Programs: Ensuring Quality and Accountability

6. FIGURE 1: ATTRIBUTES OF ONLINE PROGRAMS
STUDENTS
District’s public school enrollees Non-residents & home schoolers
TEACHERS
Local teachers Provider teachers
CURRICULUM
Locally developed Commercially developed
TIME
Synchronous school day Asynchronous anytime
PLACE
In school Anywhere
LEARNER-CONTENT INTERACTIVITY
Low High
LEARNER-TEACHER INTERACTIVITY
Low High
LEARNER-LEARNER INTERACTIVITY
Low High
LEARNER AUTONOMY
Independent, self-paced Collaborative, class-paced
FACE TIME
Never Frequent
CURRICULUM
Flexible Structured
INSTRUCTION
Flexible Structured
© David B. Glick & Associates, LLC www.glickconsulting.com
PROMISING PRACTICES 5

7. Curriculum development and course quality
Issue Variables
How can your How many courses does your program have now?
program develop How many courses are you adding each year and how are those decisions made?
online courses and
Do you mostly purchase courses, develop your own, or mix and match? What
content that are
information informs build or purchase decisions?
of high quality,
improve educational What is your course review process and schedule (i.e., how often do you evaluate
outcomes, and meet existing courses and plan new ones)?
state standards? What is your course development and review budget and staffing?
A key building block in overall program quality is the quality of the individual online courses. Online
programs are evolving in their practices in this area, as the development and maintenance of online
courses, which was once the domain of individual teachers, has evolved in many schools that now
use teams of teachers, content experts and instructional designers to ensure high-quality courses.
As the technology behind online courses and learning management systems becomes more robust,
the need for specialized skills becomes more acute. The potential for interaction and the use of
multimedia, simulations, and gaming to increase student engagement increases the range of
knowledge and expertise necessary to create online content.
The makeup of course development teams varies, but most teams consist of a project manager,
course/content writers, instructional and information designers, multimedia developers, and
copyeditors and proofreaders. When looking at an online school’s course development process,
“A good rule of thumb is to ask ‘What are the qualifications of the curriculum team?’” says Bror
Saxberg, Chief Learning Officer at K12, a provider of curriculum, technology and school services for
online programs, and the largest operator of full-time online schools in the country. “Look for a
mix of teachers, instructional designers, information designers, a solid base of research of learning
theory, multimedia experts and technologists.”
A consistent theme among online course developers is the huge difference between an online class
and a face-to-face class, in both teaching and course development. “There is a significant difference
in writing prose [narrative] and writing with an online voice,” notes Jonathan Schmalzbach, Director
of Content Development at Apex Learning. “Crafting a script that complements a multimedia
presentation of a concept is a new skill for most teachers and a difficult one to find. It’s a real coup
and pleasure to find a great classroom teacher that can translate that experience to online content
presentation, someone who understands how visuals, audio, text and other content elements
combine in a storyboarding process.”
Many online schools recognize that the act of teaching a class is quite different from the process
by which an online course is created. In a physical classroom, a teacher may divide the learning
experience into categories that include the textbook, the class lectures and activities, and additional
instructional materials. The online environment mixes and matches these components. A physical
textbook may no longer be used, increasing the importance of the way in which educational
materials and the teacher’s instruction work together. While online teachers benefit from
professional development in teaching online, creating an online course requires an additional set
6 Management and Operations of Online Programs: Ensuring Quality and Accountability

8. of skills. The Virtual High School Global Consortium (VHS), which offers extensive professional
development options to both online and classroom teachers, separates teacher professional
development in its member schools into two parts: Netcourse Instructional Methodologies for
those who are teaching an existing VHS course, and Teachers Learning Conference, which prepares
face-to-face classroom teachers to become online course developers and instructors with VHS.
KC Distance Learning takes a similar approach, with a professional development course for course
writers that includes how to incorporate student learning skills, how to develop higher order
thinking skills, and how to create a course to align with state standards.
The fact that courses are now commonly built by a team that often no longer includes the course’s
teacher creates a new set of challenges. “The course development team can build a wonderful
exercise, with excellent rubrics and teaching notes, but if the instructional department does not
respond with adequate training, the implementation of the exercise will likely go wrong and
instructional value can easily be lost,” notes education consultant John Adsit. “It may be as simple
as telling instructors not to judge student performance in threaded discussion on the number of
entries, but rather to focus on the contribution of meaningful substance, or requiring students to
define individual roles in a group exercise so that each student’s work and participation can be
evaluated separately.”
Build versus buy
Online curriculum may be developed internally by the online program, may be licensed from an
outside source, or may consist of some combination of both. Data reported by Keeping Pace with
K-12 Online Learning in 2007 based on a survey of 60 online schools suggest that the percentage
of courses that are licensed or built in-house is highly variable among online programs. Programs
license anywhere from none to all of their courses from outside providers. Indeed, the symmetry
among the percentages of courses being licensed versus being developed in-house was remarkable:
23% of online schools had licensed all of their courses, and 23% of programs had developed all of
their courses; 53% had licensed half or more of their courses, while 55% had licensed half or less of
their courses.
Reasons for using curriculum licensed from an outside source include:
The expertise of vendor development teams, including writers, instructional designers,
multimedia developers, and technologists, often exceeds the expertise within the online
program.
A wide variety of curriculum and specialty courses is difficult to produce in-house.
A program may lack staffing, funding, and/or expertise to develop and update high
production-value content.
Organizations that are focused on curriculum development often have the resources to
incorporate more extensive user testing and feedback than individual schools. They also may
be more equipped to provide regular updates, including maintenance of multimedia-based
content over generations of versions and changing technology.
Outside content ranges from individual learning objects that are incorporated into existing courses,
to full courses that are not meant to be edited, with a range of options in between. In either case,
PROMISING PRACTICES 7

9. licensing content from an outside provider does not free the online school from the responsibility of
evaluating the course content, and integrating it with successful teaching strategies.
Reasons to develop content internally include:
The need to adhere to state and district standards and greater confidence that home grown
courses will do so.
Linking content creation to teaching online in a way that involves teachers at a greater level
than licensed content may allow.
The expense of licensed curriculum, especially compared to using teachers or other staff who
are full-time.
Support for specific instructional philosophies not supported by course vendors, such as
project- or inquiry-based learning.
Course standards, quality, and revisions
Online courses are typically designed to meet national and/or state content standards. Courses and
course content licensed from outside sources are usually modified as necessary to meet state and
possibly district standards. Advanced Placement courses must meet the requirements of the College
Board. However, these standards are only a beginning.
A process of evaluation, feedback (from both students and teachers), and revision leads to
accountability and improved quality. Initial development of courses is only a start, as they should
be revised regularly to keep them current in terms of both content and technology, which includes
everything from including the latest events in a history course to making sure that links are still live
(and going to the intended information). Course revisions can take two approaches, continuous
modification or revisions on a set schedule—or a combination of both. Some programs make small
ongoing changes based on student and teacher evaluations, course failure rates, service ticket data
from support desks, or changes in external standards, and use regularly scheduled revisions to
ensure that larger changes to instructional design and multimedia content are done.
Feedback from students and teachers helps schools determine what changes need to be made to
existing courses, and provides ideas for new courses. “We create a student survey each spring and
ask students what courses they want to see created,” says Tracy Quarnstrom, Director, TRIO Wolf
Creek Distance Learning Charter School. “We also use a survey at the end of each existing course to
see what needs to be improved. With changing standards in Minnesota some classes are no longer
needed as they do not align to the [state] standards, while other topics are in growing demand.”
iNACOL’s National Standards for Online Course Quality addresses many of these issues, and others
as well. The comprehensive standards allow online programs to assure quality internally, and
demonstrate quality to outside stakeholders.
8 Management and Operations of Online Programs: Ensuring Quality and Accountability

10. iNACOL’s National Standards for Quality of Online Courses
iNACOL’s standards provide guidance in several key areas of course development and delivery. The
following list is representative of the standards, but is not comprehensive.2
Course content
The course goals and objectives are measurable and clearly state what the participants will know or
be able to do at the end of the course.
The course content and assignments are aligned with the state’s content standards or nationally
accepted content standards set for Advanced Placement courses, technology, computer science, or
other courses whose content is not included in state standards.
Issues associated with the use of copyrighted materials are addressed.
Instructional design
Course design reflects a clear understanding of student needs, and incorporates varied ways to learn
and multiple levels of mastery of the curriculum.
The course unit overview describes the objectives, activities and resources that frame the unit. It
includes a description of the activities and assignments that are central to the unit.
Each lesson includes a lesson overview, content and activities, assignments, and assessments to
provide multiple learning opportunities for students to master the content.
Technology
The course architecture permits the online teacher to add content, activities and assessments to
extend learning opportunities.
The course accommodates multiple school calendars.
Hardware, web browser and software requirements are specified.
Student assessment
Student evaluation strategies are consistent with course goals and objectives.
The course structure includes adequate and appropriate methods and procedures to assess students’
mastery of content.
Assessment strategies and tools make the student continuously aware of his/her progress in class and
mastery of the content beyond letter grades.
Assessment materials provide the flexibility to assess students in a variety of ways.
Course evaluation and management
The results of peer review and student evaluations of courses are available.
The course is evaluated regularly for effectiveness, and the findings are used as a basis for
improvement.
The course is updated periodically to ensure timeliness.
21st century skills
The course intentionally emphasizes 21st century skills in the course, including using 21st century skills
in the core subjects, 21st century content, learning and thinking skills, self-directed learning, global
awareness, and includes 21st century assessments.
For the full list of standards please see http://www.iNACOL.org/resources/nationalstandards/index.php
PROMISING PRACTICES 9

11. Teacher management
Issue Variables
How can your How many teachers does your program have, and are they primarily full-time or
program ensure part-time?
that it has the best How does your program recruit new teachers?
teachers, with the
Are your teacher orientation and professional development programs meeting the
skills and experience
needs of your faculty and goals of the program?
necessary to
successfully teach What mentoring, support, and accountability processes are used in your teacher
online? management operations?
Does your program have teacher-student communication requirements and how is
the effectiveness of the requirements measured?
Teaching is just as important in the online classroom as it is in the physical classroom. Successful
online programs use many of the following practices to ensure the highest quality faculty for their
students.
Recruiting
Because few pre-service teachers receive much training in online instruction in colleges of education,
the recruiting and initial training of highly qualified teachers to make the move to online teaching
is a critical component of success. Most programs have a clear strategy for online teacher recruiting
and training, but the criteria and techniques used vary significantly from program to program.
Most programs agree that new online teachers must 1) put their role as a facilitator of student
learning above other aspects of teaching, 2) have the ability to adapt and manage change, 3) have
a high level of content mastery, and 4) be ready to make the shift to online instruction. However,
the philosophies and techniques used to evaluate prospective online teachers vary from program
to program. Several online schools and companies, including Illinois Virtual High School, conduct
the entire recruiting process electronically, narrowing the field of candidates to those comfortable
with the use of online technology and able to demonstrate a capacity for online communication.
Other schools feel that the technology aspects are fairly easily taught, so they don’t feel the need
to vet the teachers’ technology skills as an initial step. “I’m looking for teachers with superior
teaching and communication skills, those that love to help students learn,” says Jack Babani,
Director of Instruction at Apex Learning. “We feel most any qualified teacher can be trained on the
technology components, but not content knowledge and the ability to relate to students.” Apex’s
recruiting is done through conversation and face-to-face meetings, putting the teacher behind the
student desk using role-playing and other interaction to the point where teachers often self-screen.
Some programs start the recruiting process with commercially available tests designed to gauge a
teacher’s strength in focusing on the student, rather than on curriculum or policy. Florida Virtual
School (FLVS), for example, first identifies teachers that are student-centric. Jeff Murphy, one of the
Directors of Instruction at FLVS, notes “Teachers moving to online learning must be comfortable
with the amount of change. The technology, tools and instructional methods change constantly, so
we are always probing to make sure our teachers embrace change.”
Most educators tasked with hiring teachers agree that training begins with the first recruiting
phone call or interview between the prospective teacher and the program. Several programs
10 Management and Operations of Online Programs: Ensuring Quality and Accountability

12. stress the importance of making teachers aware of the challenges of online teaching in this first
interaction. “Teachers should have no illusions of the difficulty of teaching online,” notes Bror
Saxberg, Chief Learning Officer at K12. “Most think it is as tough as the first year of teaching in
a physical classroom.” Interview techniques that help recruiters avoid potential teachers who are
more interested in the lifestyle of working from home than on tackling a new teaching challenge
are important because the desire to work from home, or the misguided belief that online teaching
requires less time than teaching in a physical school, are poor reasons to move to online teaching.
Scalability is one of the major challenges faced by rapidly growing online learning programs. K12
is facing unprecedented growth in the teaching staff at schools across the country, growing from
1,200 virtual teachers last year to 1,600 for the 2008-2009 school year. Teresa Scavulli, Senior
Director, Teacher Effectiveness at K12, believes “One of our most pressing issues is scalability. We
have to be able to move quickly to deliver educational services, almost on an as-needed basis.”
Professional development
Professional development for online teachers begins with orientation and extends throughout
the teacher’s online career. Establishing a standard training program at the point of teacher
induction gives the instructional staff a common starting point and a baseline from which to
evaluate performance. For example, the Virtual High School Global Consortium requires all new
online teachers within its nationwide network of schools to complete the Netcourse Instructional
Methodology course before they are certified to teach. This intensive 10-week graduate course
covers the skills necessary to ensure a successful start to online teaching, including the pedagogy of
online teaching, course management system basics, and online classroom management techniques
that the teacher needs to be successful.
Continued growth through professional development is critical, in particular because online learning
technology and pedagogy is changing quickly. Topics that are often part of continued professional
development include:3
Helping teachers understand how to motivate individual learners
Enhancing student interaction and understanding without visual cues
Tailoring instruction to particular learning styles
Developing Web 2.0 and 21st century skills
Modifying interactive lessons to meet individual student needs, including the needs of
disabled students
Developing heightened communication skills to enhance email correspondence and discussion
board postings, and to recognize the tone of writing and the nuances of word usage
Improving the time management skills critical for online teachers
Many online programs are now implementing extended professional development opportunities
that grow as teachers’ skills and techniques grow. For example, the VHS Global Consortium has
introduced a new five-course series, 21st Century Teaching Best Practices, that is designed for all
This list is from A National Primer on K-12 Online Learning, published by iNACOL, and was based on Essential Principles of Online
Teaching: Guidelines for Evaluating K-12 Online Teachers, Southern Regional Education Board, April 2003
PROMISING PRACTICES 11

13. levels—from classroom teachers who want to add an online component to their face-to-face class,
to experienced online teachers looking for the latest pedagogy and online methodology. These
courses explore the technology literacy and Information and Communication Technology (ICT) skills
needed for effective teaching and learning, including how face-to-face teachers can incorporate
online resources and tools into their classroom instruction, including Web 2.0 tools, WebQuests,
blogging, wikis and open educational resources (OER).
K12 professional development trainers are all top-performing, master teachers with extensive online
teaching experience. This level of expertise brings credibility with new teachers and allows them to
work as mentors and role models. Professional development is organized into four national regions,
providing greater opportunities for face-to-face training that supplements national training efforts.
K12 offers weekly online sessions for teachers grades K-8, and once a month sessions for 9-12
teachers; topics include state testing, intervention for one-to-one instruction, peer counseling, and
introducing teachers to a new student-parent community website.
Some online programs are tapping into partnerships with outside educational institutions to enhance
their internal professional development. Connections Academy, for example, has partnered with
Boise State University to tap into the university’s expertise in teacher training (along with curriculum
development and evaluation). As an increasing number of universities offer programs for existing
and pre-service online teachers, the opportunities for such partnerships will improve.
Teaching practices and evaluation
Successful online learning often is based on consistent communication between students and
teachers, and clear feedback from teachers to students. To ensure that such communication
and feedback is occurring, many online programs—and some state policies—have established
communication requirements. Examples of such guidelines include:
Student email is answered within 24 hours
The teacher initiates phone calls to each student at least monthly
A weekly progress check is recorded for each student
Student discussions are actively facilitated
Parent calls are held monthly
Weekly or monthly face-to-face meetings are held with a mentor or supervisor
Clear guidelines and expectations ease the evaluation process, helping teachers know what is
expected of them. Online teaching practice, however, is still in its early stages, with limited research
to help teachers and supervisors understand what works best. The lack of commonly accepted best
teaching practices combined with the distance at which many teachers work creates a challenge for
managers. “We don’t yet have a good sense of what is required work time and what is preparation
time that all teachers put in as an addition to their typical work day,” notes Tracy Quarnstrom,
Director, TRIO Wolf Creek Distance Learning Charter School. Other practitioners note that many
online teaching practices are still just being established and that there is little definitive research on
the specific approaches that work best for students.
12 Management and Operations of Online Programs: Ensuring Quality and Accountability

14. Although research into effective online teaching is not yet extensive, iNACOL has published a set
of online teaching standards that provide an excellent starting point. In addition to professional
development and communications standards similar to those listed above, other standards include
requiring that the teacher:
Plans, designs and incorporates strategies to encourage active learning, interaction,
participation and collaboration in the online environment
Provides online leadership in a manner that promotes student success through regular
feedback, prompt response and clear expectations
Models, guides and encourages legal, ethical, safe and healthy behavior related to
technology use
Understands and is responsive to students with special needs in the online classroom
Demonstrates competencies in creating and implementing assessments in online learning
environments in ways that assure validity and reliability of instruments and procedures
Develops and delivers assessments, projects, and assignments that meet standards-based
learning goals and assesses learning progress by measuring student achievement of learning
goals
Demonstrates competencies in using data and findings from assessments and other data
sources to modify instructional methods and content and to guide student learning
Demonstrates frequent and effective strategies that enable both teacher and students to
complete self- and pre-assessments
Mentoring and monitoring online teachers
Successful online programs often prioritize a culture of teacher collaboration and mentoring.
Although establishing clear and consistent procedures is critical to establishing accountability,
creating a sense of collaboration and mentoring fosters a collegial work environment that
encourages teachers to contribute innovative ideas. For example:
FLVS uses a co-teaching approach that mimics some of the techniques being successfully
employed in brick and mortar institutions to provide better one-to-one instruction and
teacher availability, and a team-teaching approach largely defined by the FLVS teachers
themselves. Each teacher team decides how they prefer implementing shared instructional
tasks, from monthly student phone calls to being on call for student questions, or sharing
the tasks of lesson planning and grading duties. All teachers are supported by a team of
educators including trainers, content buddies, mentors, instructional coaches, instructional
managers and instructional leaders.
K12 has established a Founder’s Club to recognize outstanding teaching efforts within its
community of teachers, and to gather and use feedback from this group to identify and
address teacher issues from across the country. Teachers are recognized based on a set of
quantitative and qualitative standards. Selected Founder’s Club teachers attend an annual
conference and work year-round to help with student retention, socialization and community
building, student achievement, parent training and student motivational issues. Feedback
from this group and other teachers led K12 to launch a study of high performing teachers.
PROMISING PRACTICES 13

15. New teachers also need to understand, as most program directors already know, that they must
be able to demonstrate accountability on a class-by-class and student-by-student basis. In the
traditional brick and mortar institution, once the classroom door is closed there is very little oversight
of the instruction and learning taking place in the room. Online learning is far more transparent and
is accessible by design, making it easy to monitor most aspects of student and course progress on a
daily basis.
Academic integrity
Online teachers and managers must also devote some of their time and practice to ensuring the
academic integrity of their classes and program. Questions and concerns regarding academic
integrity have been raised since the early days of distance learning and have continued with the
growth and acceptance of online learning. Most questions about the authenticity of student work
come from those outside the community of online learners, while most teachers and administrators
feel the issue is addressed by a number of established and proven assessment methods. Violations of
student academic integrity are not unique to the online environment, but it is still perceived to be a
challenge and a continuing important issue in online education.
Academic integrity has been defined as a commitment, even in the face of adversity, to five
fundamental values: honesty, trust, fairness, respect, and responsibility.4 Some of the threats to
academic integrity include ignorance, pressure, stress, and time management issues.
Online programs address academic integrity issues in several ways: having staff members dedicated
to the issue, ensuring consistent communication between teachers and students, requiring at least
some assessments to be proctored, and using software to check for plagiarism.
Florida Virtual School, for example, has used all of these approaches. A full-time Academic Integrity
Specialist is employed to establish and revise academic integrity policy and programs, and to work
with teachers via monthly synchronous training sessions and one-on-one when necessary. Teachers
have phone calls with students at least monthly, engaging students in a one-on-one conversation
about the content, asking probing questions that generate topical discussions. If there is an
incongruity between the student’s daily work online and the level of comprehension demonstrated
during the phone instruction, the teacher can identify potential abuses and work to correct them.
Finally, random proctored testing provides an institutionalized approach to ensure student integrity
while avoiding any perception of persecution when proctored testing is requested by a teacher.
4
The Fundamentals of Academic Integrity. The Center for Academic Integrity, Duke University. October 1999.
http://www.academicintegrity.org/fundamental.asp
14 Management and Operations of Online Programs: Ensuring Quality and Accountability

16. Student support
Issue Variables
How can your Is your program full-time, supplemental, or both?
program ensure Do most students access courses from school, home, or some other location?
that your students
Is your student orientation process sufficient and well-executed?
have the orientation,
technical support How are student progress and activity tracked, and who has access to that
and academic information?
counseling they Do your students receive robust technical support regardless of how they access
need to be their online courses?
successful? How are you supporting students with physical or learning disabilities?
Student support is an integral part of online school success. Support includes technical help (i.e.,
accessing the course, computer hardware and software, communication tools), academic issues
(course content and instruction, tutoring, counseling) and administrative support (enrollment,
personal information). Successful online programs have adopted a variety of practices to ensure top-
quality student support, including the practices described below.
Student support structures
The ways in which student support is provided varies greatly depending on whether the online
program is supplemental—meaning the student has access to support in a local physical school—or
full-time—in which case the student may be helped by a parent or other learning coach in addition
to virtual support provided by the online program.
Many supplemental online programs require that physical schools provide on-site coordinators
who assist students, while also helping adapt the school’s procedures to accommodate online
learning. IDEAL-NM, one of the newer state-led online programs, imported the philosophy and
strategy of on-site coordinators at local school partner sites from other successful online learning
programs. Site coordinators provide on-site support for students, teachers and parents, explaining
the advantages of IDEAL-NM to students, helping with student registration, monitoring academic
progress, intervening if any problems occur and in some cases working with students to provide
instruction in computer labs. “The first step in establishing a partnership with a school is to identify
the site coordinator,” says Tim Snyder, Executive Director of IDEAL-NM. “A good site coordinator is
like gold to our program. They become our knowledgeable and passionate ambassadors and set the
tone for how well the school implements the online program.” Dawn Nordine, Executive Director of
Wisconsin Virtual School—the Wisconsin Web Academy—agrees. “The support of the local mentor,
or Local Education Guide, is critical to the success of most students taking supplemental courses
online. We have increased our opportunities for face-to-face training for these folks who support
the students daily on a range of activities: placing the student, choosing the appropriate courses,
registration, teacher communication, and policy development.” The Idaho Digital Learning Academy
(IDLA) has taken the need for strong local support a step further, by reducing the course fees for
districts that have had a site coordinator go through an IDLA professional development course.
PROMISING PRACTICES 15

17. Enrollment and orientation
The first contact between a prospective online student and the online school is an important step in
ensuring a successful transition to online learning. Established procedures help guide online learning
staff in how to mentor the student through the process of enrolling and participating in online
courses, covering issues such as:
An initial checklist of points to cover with students and parents during the initial period
A pre-enrollment survey to challenge students’ preconceptions of online learning and to
determine their level of readiness for this new modality
A student orientation course before the first academic course to set performance
expectations, familiarize the student with the learning management system and identify any
technical support issues
A review of online learning policies to cover grading requirements, student discipline, and
warning and probation policies
Technical support
Student access to robust technical support relieves one of the key barriers to student success in
online learning while taking a significant burden off teachers. To reduce initial technical support
calls, many programs use automated checks of bandwidth, versions of Flash, Acrobat, and Java, and
other plug-ins required of students. Tutorials and online orientation sessions familiarize students with
the learning management system to assist in a smoother transition to the online environment. Many
programs make help desk support available 24/7 both by phone and virtually, and many have or are
instituting service ticket systems to track more carefully student support communication and results.
In some cases, students are taking support into their own hands. SHOCK (Students Helping Others
Collect Knowledge) is an innovative new student support program at Florida Virtual School that was
envisioned and executed by past students to provide peer-to-peer support to help students with
everything from how to navigate and use the learning management system, to how to study for a
math exam.
Academic support
Technical support is just one component of helping students learn online; the other key component
is academic support. This may include, but is certainly not limited to, the following set of tools and
techniques to ensure a smooth transition to online learning. In some programs, these are handled by
teachers, while others have features for such support built into their learning management systems.
In other programs, these are functions performed by either face-to-face or virtual non-teaching staff.
Assessments of student progress at regular intervals
Support materials like student handbooks
Guidelines for conduct (both students and teachers)
Organization of students into groups/cohorts
Tutoring services
Pre-course tests that measure a student’s readiness for the online learning experience
16 Management and Operations of Online Programs: Ensuring Quality and Accountability

18. Required amount of course access
time and how that differs from
time on task Working with local school districts
Expectation of the parent and
Most schools are excited by the possibilities
family roles in the student’s
of online learning, but concerns arise once the
experience; parents’ role in
reality of implementing online instruction sets
monitoring course progress, in; how do we make this happen? The Hamilton
conference calls with teachers, County Virtual School (TN) and IDEAL-NM
face-to-face opportunities for distribute a handbook to districts ready to have
students and parents their students take supplemental online courses.
The handbook gives school administrators and
Communication is the key element of on-site coordinators the procedures, application
student support. Students should never documents and policies necessary to get
feel that they don’t know where to the program off the ground. The IDEAL-NM
turn with a question or problem, and handbook, for example, outlines a step-by-step
should be able to easily reach their online process including:
teacher, school facilitator, learning coach, Defining the roles and responsibilities of
or other source of assistance. on-site people with “getting started tips”
for each position
Counseling and mentoring Explaining the costs and funding of the
program
Physical schools provide counseling and
mentoring to students. Online schools are Describing the learning management
moving in this direction as well, either by system and other technology
instituting online counseling support for Providing program policies, from
students, or extending the training for acceptable use and attendance policies,
on-site coordinators who often perform to how the program and partners should
many of the tasks of a traditional school handle conduct and discipline problems
counselor. and withdrawals.
This handbook, and the procedures and
School-based counseling comes with a policies it outlines, eases concerns for school
heavy load of administrative duties (credit administrators, demonstrates the level of
analysis, senior credits for graduation, support IDEAL-NM provides, and makes
community service hours) that limit program implementation easier by providing a
blueprint to success.
personal contact with students and
provide little opportunity to be proactive.
In supplemental online learning situations,
the online program may be supporting
the district’s counseling efforts, so the
online counselor is freed to focus on
students: establishing regular communications, leading students to online resources published by
the online program, answering general questions, or encouraging the student to look for resources
outside the school system when necessary. The site facilitator in a supplemental program may or
may not have roles defined in tandem with the physical school counselor—in some cases the two
roles may be taken on by the same person.
PROMISING PRACTICES 17

19. Some supplemental online programs, and most full-time online schools, have full-time online
counselors. At FLVS, the counselor’s role begins with student recruitment and orientation and
includes responsibilities ranging from monitoring student progress to collaborating with online
teachers to mediate performance and behavioral issues. Students are often more expressive with
their online counselor, comforted by the relative anonymity of email and phone communication as
compared to face-to-face meetings that can be intimidating for students. “The online environment
allows for more one-on-one connection than I had as a counselor in a physical school,” according
to Patty Cordones, Online Counselor for Florida Virtual School. “Student needs range from those
as casual as a complaint about a course being too rigorous to something as profound as a suicide
intervention. As counselors, we’re trained to follow defined procedures when serious problems arise;
call the parent, the teacher and district, and find help for that student.”
Technology management
Issue Variables/Elements
How can your program Does your program have a clear set of functional criteria and operational
develop a technology objectives for the learning management system (LMS) application?
strategy and systems to Does the LMS integrate with the program’s student information system (SIS),
improve student and teacher and is the right student data being captured and used effectively?
learning, and provide data
Does your program have an annual evaluation of its technology needs and
for the daily management
performance to keep pace with the rapid rate of technological change?
and oversight of staff, while
anticipating change and the Can existing technology be scaled to accommodate program growth and
need for innovation? adapted to new instructional technology tools?
While online educators often point out that teachers, and not computers, are at the heart of online
learning, technology systems are clearly an important component of an online school. Computer
hardware and software are essentially the facilities of an online school, much as classrooms and
buildings are the facilities of a physical school. These tools not only provide information and data to
manage the program, but also help teachers become more innovative and effective at their jobs.
Following are some of the practices successful online programs employ to ensure that their
technology systems contribute to overall program quality.
The role of LMS and SIS technology
Online education is first and foremost based on the interaction between the student and the
teacher, but technology plays a significant supporting role in delivering instruction and an integral
role in providing accountability and management tools. The technology at the heart of an online
school is the learning management system (LMS)—the set of tools that houses course content and
provides the framework for communication between students, teachers, and parents. The student
information system (SIS), which manages student data, is the other main software component.
Teaching online through an LMS presents several advantages compared to a physical classroom,
including:
18 Management and Operations of Online Programs: Ensuring Quality and Accountability

20. Monitoring student performance, including observing individual student trends quickly using
real-time online data; also tracking of attendance to meet state requirements for funding;
and to meet regulatory demands
Organizing student learning by age groups; younger students may have difficulty sitting at
a computer for hours, while older students can make better use of video, animations, and
simulations
Allowing for quick updating of curriculum
Scheduling and tracking for families of full-time students, allowing easy monitoring of
assignments and tasks completed
Adapting instruction to adjust to the student based on the difficulties being encountered
Keeping up with improvements in technology infrastructure
Experienced online programs commonly make a point to “know your core competency and
outsource the rest,” noting that they should not get too caught up in technological change and
lose their focus on education. “VHS Global Consortium stays focused on our core competency
and mission, to educate students,” says Liz Pape, CEO. “We are not a data support center or an
LMS developer, but it’s critical that we maintain strategic management staff to closely monitor
our vendors and ensure the technology we employ from outside the organization provides the
information and data we need to manage the program and manage the entire organization.”
Programs must evaluate new technology and recognize when the tools are largely “bells and
whistles” and when they truly advance instructional practice and value. It is far too easy to get
caught up in the possibilities of an instructional technology before it is reviewed and tested by those
with the daily experience to judge its merits.
Equity and privacy issues impact technology oversight decisions and processes. Technology
departments must establish policies to protect student data and information, as well as meet
standards for access by all students regardless of connectivity or disabilities.
Regardless of the learning management system and administrative software in place, programs
must plan for future flexibility and growth, not just daily operations of the existing systems. “Be
prepared for change in the world of technology, meet regularly as a team and build in professional
development and training for the entire technology staff,” says Jonathan Beckham, FLVS Senior
Manager Information Services.
PROMISING PRACTICES 19

21. Program evaluation
Issue Variables
Is your program What is your program’s mission; does the mission include specific student
meeting its mission populations?
and goals as well as Are student outcomes meeting program and state expectations? Can improved
the expectations of student outcomes be demonstrated?
the stakeholders and
Does your organization conduct an annual program evaluation?
the community?
How satisfied are stakeholders, including students, schools, and parents, with their
experience with the program?
Because online learning is still relatively new, online schools sometimes have to demonstrate quality
and results in ways that go beyond the requirements that physical schools meet. Supplemental
programs, in particular, face the challenge of not being evaluated in the same way as public schools,
because they are not responsible for their students’ state assessment scores (in most cases). One
way that online schools address quality and performance concerns is by commissioning regular
program evaluations.
Internal and external evaluations
Program evaluations fall into two categories: internal (conducted by the program staff) or external
(conducted by someone outside the organization). Internal evaluations have the advantage of
timeliness, as they can generally be scheduled conveniently at any time during the year, and staff
that have a thorough knowledge of the program being evaluated.
External evaluations bring a fresh look at a program from someone removed from the pressure of
stakeholders or program staff. There are no personal relationships to cloud the examination of the
program, and an outside perspective may bring flaws to light that go unseen by staff too close to
the situation to recognize the problems. Also, external evaluations often carry greater validity with
stakeholders. However, the cost of external evaluations can be high and the process takes additional
time to bring the evaluator up to speed on the processes and metrics of the program being
evaluated.5
Pieces of the evaluation puzzle
Although the specifics of an evaluation vary significantly, an evaluation usually starts with an
examination of the program’s mission to determine whether it is meeting its organizational goals.
If the goal of an online program is to give students more opportunities for recovering course
credits in order to graduate, then two evaluation measures could be the number of credit recovery
courses being offered, and the graduation rate for students in those courses. If the mission of the
online program is to increase 21st century learning opportunities for a district’s students, possible
evaluation criteria include 1) the number of new courses available to students; 2) the increase in
student proficiency in use of Web 2.0 tools; and 3) the increase in teachers’ use of Web 2.0 tools in
classroom and online instruction.
5
Evaluation in Online Learning, L. Pape, M. Wicks, C. Brown, and W. P. Dickson, in Keeping Pace with K-12 Online Learning, 2008,
www.kpk12.com; further examples are from this source as well
20 Management and Operations of Online Programs: Ensuring Quality and Accountability

22. Evolving Models for Performance Data Systems: Florida Virtual School’s
Virtual School Administrator
Florida Virtual School (FLVS) is not only the largest online program in the country, but it is
also the only program whose funding is based primarily on student achievement instead of
measures such as seat time, which are common across K-12 education. With funding based on
student achievement, consistent tracking of course progress based on competency becomes
critically important not just for students and teachers, but for administrators as well. The
FLVS Virtual School Administrator (VSA), a custom program integrated with the program’s
course management system, has been instrumental in helping FLVS meet this challenge.
VSA provides sophisticated functionality to measure ongoing competency throughout each
course and performance levels for each student, and consolidates data from the LMS for use
by teachers and administrators. The technology staff works closely with instructional and
course development personnel to track data and create reports that meet the needs of each
department. Dashboards have been created to provide at-a-glance insight into a student’s
course progress, grade status, a teacher’s monthly phone call log, and responsiveness to
email, among other data points. All of the operational departments of FLVS participate in
defining the data and applications they need to improve their effectiveness and provide a
competency-based learning experience for students. The FLVS technology group is working to
add predictive analysis capability to the existing system, using historical data and patterns
to predict and improve student performance.
Although in-house software development is sometimes the best strategic path, FLVS wants
to spend its time educating students—not responding to technology-related help tickets
from students and teachers. While the development of VSA has been handled in-house to
accommodate the needs of the stakeholders, FLVS made the strategic decision to outsource
student help desk and first-line teacher technology support. With the large volume of help
tickets encountered by FLVS, a software tool was selected that routes questions and problems
to the appropriate technical staff for quick response and generates metrics on the instances
of help tickets, the types of problems, how quickly the problems were resolved and who had
to be involved in the solutions. Jonathan Beckham, Senior Manager Information Services
at FLVS, stresses, “You are not done with the decision on which vendor to use and which
software package to buy. With all outsourcing, it is critical for the organization to maintain
technical staff to manage and monitor the work of the vendor. Use outsourcing when available
and reliable, but be vigilant and establish internal oversight of and management of vendors.”
Evaluations are commonly based on one or more of stakeholder surveys, outcomes data, and
reviews of internal processes.
Surveys may be done of students, parents, teachers, educators in school districts using
supplemental online courses, and other stakeholders. They may be done at multiple times
of year; for example, students may be surveyed every semester while parents are contacted
once per year.
PROMISING PRACTICES 21

23. Outcomes data vary based on program type. Supplemental programs often rely on course
completion rates and results of Advanced Placement exams, while full-time programs can
report data of state assessments and other measures common to all public schools in a state.
Internal processes such as course development may be benchmarked against other
programs, or standards such as those published by iNACOL. In addition, the evaluation may
report financial information, staffing levels, and similar organizational metrics.
Conclusion
Online learning promises cost-effective solutions to the challenges education leaders face in many
areas, particularly in difficult economic times. States and districts are increasingly turning to online
programs to expand educational opportunity, equity, and access while individualizing learning
options for students. This growth in online learning requires that school managers be able to
effectively operate and assess their programs.
While most online programs deliver quality curriculum, effective teaching, and improved student
outcomes, such positive outcomes are not assured. They are instead a result of forethought and
diligent management, starting with defining and planning the online program, and extending
through all aspects of management and operations.
The growth of online learning programs, the demands of expanding course offerings, and the
need to service different types of students and deliver learning in a variety of formats are just a
few elements that illustrate the complexity of running a quality online learning program. Although
many existing online learning programs started as an individual vision, created and fostered by a
small, dedicated team of teachers and administrators, most programs now recognize the need for
processes and procedures that ensure program quality and accountability.
Managing growth, and keeping up-to-date, are two of the issues that online programs commonly
face. “One of the next challenges in program management is staying current: currency in teaching
skills, currency in course design standards, and currency in technology strategies as broadband
access and new online tools bring new instructional opportunities to bear,” says Liz Pape, CEO, VHS
Global Consortium. “Ubiquitous computing will be demanded by students and improve teaching
and learning, but it will require constant evaluation and integration into existing programs.”
Online learning is already improving student outcomes, and holds the potential to be a truly
transformative element of education in the years ahead. Clayton Christensen, Harvard Business
School Professor and one of the authors of Disrupting Class: How Disruptive Innovation Will Change
the Way the World Learns, predicts that in about a decade, half of all high school courses will be
online. But previous trends do not guarantee future growth, and online educators recognize that
their programs face far greater scrutiny than physical schools.
Regulators and policymakers at the state and federal level often grapple to find the appropriate
balance between mandating quality and accountability on the one hand, and allowing room for
innovation on the other. Programs that can demonstrate a sensible approach to ensuring quality
content in their courses, highly qualified and well-trained teachers, comprehensive student support
22 Management and Operations of Online Programs: Ensuring Quality and Accountability

24. services and forward-looking technology systems can help guide constructive policy and regulatory
“Ensuring quality in a fast-growing enterprise like online learning is like upgrading the engine on a jetliner
while it is in flight,” says Mickey Revenaugh, Vice President for State Relations at Connections Academy.
“It’s an enormous challenge— but one that virtual program managers must embrace wholeheartedly. If we
as online educators don’t do all we can voluntarily to ensure that we have every possible quality system in
place, we can be certain that policy-makers and regulators will attempt to do the job for us.”
PROMISING PRACTICES 23

25. Appendix A
ONLINE LEARNING:
THE OPPORTUNITIES IT CAN PROVIDE; THE PROBLEMS IT CAN SOLVE
Below is a list of opportunities that online learning can help provide for your teachers and students and
problems that online learning can help solve. Please prioritize them by circling the appropriate number.
Very high High Low Very low
priority priority priority priority
1. Provide methods for differentiating instruction for groups and 4 3 2 1
individuals at all grade levels.
2. Bring more Advanced Placement courses to more students. 4 3 2 1
3. Effectively meet the needs of homebound & hospital bound 4 3 2 1
students.
4. Increase the number of electives available to students. 4 3 2 1
5. Help alternative learning and credit-make up students complete 4 3 2 1
necessary coursework.
6. Engage unengaged students through technology and 4 3 2 1
independence.
7. Allow students who are socially uncomfortable to focus on 4 3 2 1
academics.
8. Reduce class sizes by providing more options for students. 4 3 2 1
9. Allow students to more easily blend high school and 4 3 2 1
post-secondary options.
10. Allow students to communicate with a wide range of people, 4 3 2 1
potentially from around the world.
11. Bring world-class resources and timely information to 4 3 2 1
coursework.
12. Allow students to more easily blend school with duties at home 4 3 2 1
or work.
13. Provide individualized instruction and unique learning 4 3 2 1
opportunities.
14. Increase teachers’ familiarity with instructional technologies 4 3 2 1
and how to use them in the classroom.
15. Help students use technology as an essential tool for learning. 4 3 2 1
16. Allow students who don’t like school to succeed academically. 4 3 2 1
17. Fill curriculum gaps for any grade or ability level. 4 3 2 1
18. Help to fill in areas where available teaching staff is minimal or 4 3 2 1
requires additional instructional support.
19. Provide a cost-effective way of providing low-enrollment 4 3 2 1
courses.
20. Allow school-day and calendar flexibility for students. 4 3 2 1
21. Allow school-day and calendar flexibility for teachers. 4 3 2 1
22. Concretely demonstrate how technology enhances education 4 3 2 1
to develop support for technology initiatives.
23. Archive classroom experiences for students, parents or 4 3 2 1
accountability.
24. Modernize curriculum and save money by replacing some or 4 3 2 1
all textbook purchases.
25. Other 4 3 2 1
26. Other 4 3 2 1
27. Other 4 3 2 1
© David B. Glick & Associates, LLC www.glickconsulting.com
24 Management and Operations of Online Programs: Ensuring Quality and Accountability

26. Appendix B: Resources
The International Association for K-12 Online Learning (iNACOL) released two important volumes to
aid online programs in establishing guidelines for online teaching and course quality. The National
Standards for Quality Online Teaching is a publication that provides states, districts, online programs,
and other organizations with a set of quality guidelines for online teaching. National Standards of
Quality for Online Courses provides standards selected on the basis of a research review and survey
of online course quality criteria. These quality standards were evaluated and assembled into an
easy to use document for evaluating online courses with common benchmarks. Both documents
can be downloaded at the iNACOL web site, http://www.iNACOL.org/nationalstandards/index.
Colorado Online Learning posts a number of helpful resources under their Quality Assurance
Program (QAP), including the full text of the QAP document, http://www.col.k12.co.us/aboutus/
The VHS Global Consortium program evaluations are available through its website at http://www.
National Education Technology Plan, US Dept of Education, http://www.ed.gov/about/offices/list/os/
technology/plan/2004/site/edlite-default.html, retrieved January 13, 2009.
Evaluating Online Programs, US Dept of Education, http://www.ed.gov/admins/lead/academic/
evalonline/report_pg29.html, retrieved January 14, 2009.
Essential Principles of High-Quality Online Teaching: Guidelines for Evaluating K-12 Online Teachers,
Southern Regional Education Board (SREB), http://www.sreb.org/programs/edtech/pubs/PDF/
Essential_Principles.pdf, retrieved January 13, 2009.
A National Primer for K-12 Online Learning , iNACOL, http://www.iNACOL.org/resources/docs/
national_report.pdf, retrieved January 12, 2009.
National Standards of Quality for Online Courses, iNACOL, http://www.iNACOL.org/resources/
nationalstandards/, retrieved January 13, 2009.
Keeping Pace with K-12 Online Learning: A Review of State-Level Policy and Practice, 2008, http://
Trujillo Commission on Online Education, Donnell-Kay Foundation, http://www.iNACOL.org/
resources/docs/TrujilloCommissionOnlineEducationFinalReport-2-15-2007.pdf, retrieved January 15,
PROMISING PRACTICES 25

27.

28. TOLL-FREE888.95.NACOL (888.956.2265) DIRECT 703.752.6216 fax 703.752.6201
email info@inacol.org web www.inacol.org
mail 1934 Old Gallows Road, Suite 350 Vienna, VA 22182-4040

Book a Trial Class

Download