This article has been retrieved times since September 23, 1997.

Comprehensive Study of Factors Impacting Perceived Quality in School Organizations:
Findings from research on quality assessment in Iowa school districts

William K. Poston Jr.
Associate Professor
Department of Professional Studies
Iowa State University

 
 

Abstract

            This paper presents the findings of studies conducted at Iowa State University of public schools in Iowa in the area of perceived quality assessment.  Demographic characteristics of the respondents on the Perceived Quality Assessment Instrument from forty-four school districts were described by position, home annual income, gender, age, level of education, and years experience in current or a similar job.
            The research project undertaken incorporated several studies of quality improvement characteristics of public schools.  The project resulted in a compendium of coordinated research aimed at learning more about the relationships and effects of quality improvement efforts with other factors of school district operations.
            The individual study components (doctoral dissertations) focused on the following issues:

• Assessment of quality improvement climate in community colleges (Bax, 1994).
• Teachers' perceptions of training programs and their relationships to total district  perceptions of quality management (Johnson, 1995).
• Performance-based pay of chief executive officers and effects upon quality improvement processes in school organizations (Behounek, 1996).
• Financial characteristics of school organizations and relationships to quality management factors (Kirchoff, 1996).

            This series of studies conducted over a period beginning in 1993 and culminating in 1996, was designed to assess the perceptions of school district stakeholders about the quality of their school district in terms of the seven quality dimensions of the Baldrige Award criteria.  The instrument, in two versions, focused on the following Baldrige Award areas:

1.  Leadership
2.  Information and Analysis
3.  Strategic Quality Planning
4.  Human Resource Development and Management
5.  Management of Process Quality
6.  Quality and Operational Results
7.  Client Focus and Satisfaction

            Two other instruments were developed for use as a part of the -- the Staff Development and the Executive Compensation Questionnaire was developed and used in the study.  Financial information was obtained from government records.
            A determination was also sought to establish if the PQAI  differentiated in terms of quality between high ranking and low ranking school districts.  Inferential statistics established not only a significant difference between the high and low groups' quality effectiveness index, but there were also significant differences between the groups in each of the seven dimensions or sub-areas of the PQAI instrument.
            A significant positive relationship was found between the perceived quality of district staff development and the perceived quality effectiveness index of the districts.   Also, differences between performance-related and situational-related (nonperformance)  factors were evidenced between board presidents and superintendents and a weak inverse relationship was found between performance-based compensation support and the perceived quality of the systems.   Significant differences were found between the high and low QEI groups in two areas -- revenues per pupil in the leadership sub-area, and transportation cost per pupil in the information and analysis category.  No correlation was found between the sample schools financial characteristics and their any PQAI  rating area with one exception -- transportation cost per pupil and information and analysis.

Introduction

Fragmentation is the process by which we take things apart and study the parts as a means to  understanding the whole.  This is an appropriate model for mechanical systems, but for organizations, it is the interaction among the parts that makes things happen (Graff, 1995).

            The quality improvement emphasis of the private sector has been permeating gradually into the public schools for over a decade. The direction school organizations have been heading has been modified during this period in attempts to improve the quality of schooling, the value of educational programs, and the accomplishments of  learners.  Many changes were effected in schools in America during the aforementioned period, and support for sustained efforts to reform education has come from federal, state, and local levels of  government (Lezotte, 1994).  Once focused exclusively on manufacturing industries, quality improvement strategies have been developed and implemented in many service organizations, including school systems.
            Quality improvement has become nearly a standard fixture of organizational development and staff training in schools, and the importance given to various principles of quality improvement has spawned considerable interest in the suitability of such strategies for school systems.  Suddenly, policy officials are asking whether  quality improvement strategies make sense for school organizations, in which terms such as "product" or "customer" have imprecise definition.
            School organizations' focus on quality improvement strategies is an outgrowth of several policy efforts in the United States and other educationally progressive countries.  These policy efforts indicate that everyone must become "champions of change" (Rowe, 1994).  Old  leadership ideas won't work anymore, and new leadership principles must recognize a number of factors:

•   Organizational hierarchies don't work
•   Vision has power
•   Shared values create alignment
•   Successful management incorporates coaching, vision, and facilitation
•   Feedback on performance is crucial for improvement over time

            In order to foster organizational change, educational institutions have gathered in ideas from systems theory, psychology, management theory, human-resource and organizational development, statistical process control, and human synergy.  All of these ideas, in many guises and combinations, aim to remake organizations so they become more focused, disciplined, quick-footed, humane and competitive (Marchese, 1993).  Many of these efforts are aimed at reducing the level of criticism toward schools and traditional types of organizational operations  (Bradley, 1993)
            Regardless of motivation, school systems have undertaken a number of initiatives to improve classroom instruction, academic units, staff attitudes, and financial costs (Teeter and Lozier, 1993).  A plethora of school improvements and reforms have been designed and implemented with little examination of the net results or effect.  Often, school reforms and improvements are focused on a subordinate part of the system rather than on system-wide change (U.S. General Accounting Office, 1993).  Specific effects of quality improvement efforts have not been adequately anchored in research findings to determine their suitability and usefulness, and the need is widely apparent for identifying how and under what circumstances quality is manifested in school systems
            This medley of studies examines relationships and differences among groups, factors, and conditions associated with quality improvement in schools.  Specifically, this paper summarizes the problems, procedures, findings, and conclusions of four doctoral studies conducted under the supervision of Dr. William K. Poston Jr., Associate Professor of Professional Studies, Iowa State University.  The four studies addressed relationships and inferences in the areas of job satisfaction and work climate, staff development and training, compensation formats and incentives, and financial characteristics.  The doctoral students involved in these four study areas included Dr. Rashid M. Bax, Malaysia Ministry of Education, Dr. Pamela D. Johnson, West Des Moines (Iowa) Community School District, Dr. Thomas E. Behounek, Milo (Minnesota) Public School District, and Dr. Joseph E. Kirchoff, Manchester (Iowa) Community School District.  A number of additional studies are currently underway in the areas of strategic planning, facilities and productivity, and student achievement by several other graduate students at Iowa State University.

Definitions of Terms

            Quality improvement is characterized by specific terminology.  Terms utilized in this study are defined below to provide clarity and understanding of their use in these studies:
 

Executive Compensation:  The sum paid to the chief executive officer of an educational institution and the underlying rationale for its determination, including results of performance or situational circumstances.

Job Satisfaction:  One's perceived emotional state expressed as a point on a scale representing the degree of positivity or pleasure resulting from the appraisal of one's job or one's experience (Locke, 1976).

Malcolm Baldrige Award:  A national award presented by the United States Government to businesses, companies, or organizations for their demonstration of excellence and exemplification of quality principles (Schenkat, 1993).  It is similar to the Deming Award in Japan at least in intent.

Organizational Commitment:  Relative strength of an individual's identification with, and involvement in, a particular organization.  Conceptually, it can be characterized by at least three factors: (1) a strong belief in, and acceptance of, the organization's goals and values; (2) a willingness to exert considerable effort on behalf of the organization, and (3) a strong desire to maintain membership in the organization (Porter, Steers, Mowday, and Boulian, 1974, p. 604).

Perceived Quality Assessment Instrument:  A questionnaire developed collaboratively by William K. Poston Jr. and Rashid M. Bax at Iowa State University designed to elicit perceptions of organizational stakeholders on seven dimensions of quality improvement modeled after the Baldrige Award criteria.

Perceived Quality Effectiveness Index:  A ratio or score produced by dividing the stakeholders' perceptions of the current status response score by the desired status response score, expressed as a decimal.  Ratios less than one (1) indicate need for improvement; ratios greater than one (1) indicate exceeding expectations.

Quality Improvement (synonyms: quality management, continuous quality improvement, or total quality management): a customer-focused strategic and systematic approach to continuous performance improvement (in an organization)  (Vincoli, 1991, p. 28).

Staff Development or Professional Development:  Activities of school organizations that seek to prepare employees for improved performance in their present or future duties and responsibilities.

Staff Development Questionnaire:  An instrument developed for the purpose of measuring the perceived quality of staff development activities in an educational organization (Johnson, 1995).

Systems Theory:  A practice or intention to view and work with organizations as an interdependent set of components including purpose, people, methods, environment, materials and other factors that influence the organization's functioning and activities and that affect and interrelate with all other factors.

 

Purposes and Rationale for the Study

            There were three main purposes for conducting this series of studies.  The first purpose of these studies was to determine if the Perceived Quality Assessment Instrument (commonly referred to as the PQAI) designed and developed at Iowa State University was effective in measuring factors commonly associated with excellence, efficacy, and effectiveness in school organizations.  The second purpose was to identify the extent of any relationships and the nature of any associations among variables measured by the PQAI and other measures used as a part of the studies.
            Finally, the purpose was to find whether or not operations and management actions in schools have an impact or influence upon the perceived level of quality within the system.  Understandings obtained from these data should help school systems seeking improvement of quality to see more clearly what gaps exist among present and desired organizational conditions and what part the studied variables play in a systems approach for school organizations.  The studies also attempted to demonstrate how management decisions might close the gap between the "current" and "ideal" status of factors as measured on the PQAI.
 

Research Questions for the Study

            Questions relevant to the studies were formulated and postulated in four areas -- general perceptions of quality management, staff development, executive compensation, and financial factors.  The questions addressed are listed below

1.  Perceptions of Quality Management Assessment:

• What are the perceived current and ideal levels of quality management and the quality effectiveness index (ratio) between the two levels in the selected districts?
• What components of the perceived quality assessment instrument have the greatest impact on the overall quality rating of the selected districts?
• What value can be ascribed to the Perceived Quality Assessment Instrument in reliability and validity in measuring system quality?

2.  Staff Development Questions:
 

• What are perceptions of quality of staff development programs in the selected districts?
• Is there a relationship between the perceived quality of district staff development and the perceived level of quality management in the selected districts?
• Is there a relationship between ratings of perceived levels of staff development programs and the seven dimensions of district quality?
• How do districts rated highest and lowest in perceived quality of staff development differ or compare on the perceived quality effectiveness index?

3.  Executive Compensation Questions
 

• What factors are used and which are preferred in the compensation of the chief executive officer of school districts by superintendents and governing board presidents and how do these two groups differ in choice of factors determining compensation?
• How do methods used and preferred in determining compensation for the chief executive relate to levels of quality in the system as measured by the Perceived Quality Effectiveness Index?

4.  Financial Factors Questions
 

• To what extent are there differences between districts with perceived high and low quality on the Perceived Quality Assessment Instrument in terms of financial characteristics?
•   Is there a relationship between the financial stability or soundness of school districts and their perceived quality effectiveness index or sub-scales of the Perceived Quality Assessment Instrument?

Hypotheses and Assumptions of the Studies

            All hypotheses were constructed and analyzed in null form for statistical analyses.
            Several assumptions supported the studies, including the following:

• The population of the study was representative of the districts in the state of Iowa, and respondents were representative of the total population of the systems involved in the studies.
• Respondents understood the content and direction of instrumentation.
• Subjects voluntarily participated in the studies by completing instrumentation.
• The measured perceptions accurately reflected actual levels of organizational quality.
• Respondents were knowledgeable about their school system, and responded accurately and honestly to all instrumentation.

Rationale for Quality Improvement in School Organizations

            Continuous quality improvement, also inaccurately called total quality management, has been gathering momentum in the United States and elsewhere.  The systems thinking approach was led by W. Edwards Deming, a Sioux City, Iowa native, before his death in 1993, and it has continued strongly since.  The "new philosophy" has affected businesses, industries, government, and educational institutions (Brown, 1992).  Literature on quality improvement has mushroomed in the past several years with application for school organizations.  Deming often wrote and spoke of "continuous improvement" with the goal of quality, so the term "continuous quality improvement" is frequently used in discussions about school organization improvement, reform, or transformation (Johnson, 1995).
            Education is in need of dramatic change, according to recent critics.  Transformation has been called for in government, industry, and education in the United States, and such change must "be a change of state, metamorphosis, not mere patchwork on the present system of management (Deming, 1990).
            Specific approaches for improvement are less than clear and distinct.  According to Minnesota's Lieutenant Governor, Joanell Drystad, fragmented, individual attempts at system repair have not succeeded, and the only answer is fundamental, systematic transformation.  In an address to over 2000 teachers, administrators, students, policy-makers, and business partners from 26 of the United States, Finland, and Canada, the call was made to "accelerate local transformation efforts in order to meet the national goals by the year 2000 by deploying quality quickly.  By sharing and working together through the total quality systems approach, we can improve our nation's schools, provide better learning options to our students, and ensure a world-class work force"  (Drystad, 1994).
            A study group of forty-four school districts was identified as the population for the research project.  The Perceived Quality Assessment Instrument was administered to the group of school districts.  The criteria used in development of the specific items on the instrument (see Appendix for sample of the instrument) were drawn from research literature and documented successful institutional practice.  The criteria reflect a number of factors that focus upon the development of educational excellence.
            Continuous improvement is an organizational behavior grounded in several important characteristics -- clear goals, mission, and organizational expectations, defined direction for the design and delivery of superior teaching and learning, continuous focus on results and the use of feedback in decision-making, equitable and consistent connections among all organizational components for all clients and stakeholders, and efficient and effective use of resources (Frase, English, and Poston, 1995).  These factors form a foundation for the assessment of organizational functioning, and comprise a comprehensive assessment strategy.  The characteristics of such a strategy should include the following (US Dept. Commerce, 1995):

• Clear ties between what is assessed and the school system's objectives, particularly in what clients are to obtain (learning, services, etc.) or gain from the organization.
• A focus on improvement -- built upon a definition of student performance, faculty and staff capabilities, and program performance.
• Assessment as "embedded and ongoing" that is curriculum-based, criterion-referenced, and aimed at fostering improved understanding and accomplishment of goals and requirements.
• Clear guidelines as to how assessment results will be used.
• An ongoing method to evaluate the evaluation process to improve the connection between goals and client success.

            For effective school improvement to occur, significant changes will have to be made in how school systems function.  Many facets of school operations will have to be modified, but serious commitment calls for an understanding and implementation of quality management.  The past success of quality management in business and industry raise the possibility of applying its principles to education (Teigland, 1994).  It is important that the highest levels of leadership provide direction for quality improvement to occur (Bax, 1994; Walton, 1986).  School administrators must seriously consider quality improvement as one option for bringing about much needed change and improvement in school organizations (Teigland, 1995).
 

The Quality Improvement Studies

            Despite its recency in use as an organizational development tool, quality improvement in schools does not have a long history of research in its efficacy and impact upon commonly held measures of school organizational characteristics.  At Iowa State University, instrumentation was developed for use in measurement of the status of perceived quality within a school organization.  The assessment tool, the Perceived Quality Assessment Instrument (PQAI) was structured to determine the perceived level of quality of a school system based upon selected Baldrige Award criteria.

Baldrige Award Criteria
 
            This group of impact studies involved an adaptation of the Malcolm Baldrige Award criteria.  The Baldrige Award, begun in the United States, was designed to recognize corporations for excellence in achieving quality.  The Malcolm Baldrige National Quality Award (Baldrige Award) was established in 1987 through legislation (P.L. 100-107).  The purposes of the Baldrige Award were threefold:

• To promote awareness of the importance of quality improvement to the national economy;
• To recognize organizations which have made substantial improvements in products, services, and overall competitive performance; and
• To foster sharing of best practices information among U.S. organizations.

            Eligibility for the Baldrige Award was initially open only to for-profit organizations. However, recent developments indicate that eligibility might be extended in the future to not-for-profit organizations.
            The Baldrige Award Program strategy consists of two parts: (1) conceptual and (2) institutional.  The conceptual part of the strategy involves the creation of consensus criteria which project clear values, set high standards, focus on key requirements for organizational excellence, and create means for assessing progress relative to these requirements. The institutional part of the strategy involves use of the Criteria as a basis for consistent communications within and among organizations of all types. Such communications stimulate broad involvement and cooperation, and afford a meaningful and consistent basis for sharing information. An important part of the communications is the sharing of information by Baldrige Award recipients (U.S. Dept. of Commerce, 1995).
            Through the Baldrige Award, rigorous criteria were created to evaluate applicants for the Award. The Baldrige Award Criteria, based upon a set of core values and concepts, focus on key requirements for organizational excellence. These requirements are incorporated in a seven-part Criteria framework. Accompanying this framework is a set of Scoring Guidelines which permit evaluation of performance relative to the detailed Criteria. The evaluation leads to a feedback report--a summary of strengths and areas for improvement. All Baldrige Award applicants receive a feedback report.
            The Baldrige Award Criteria and Scoring Guidelines have led to a number of key developments:

• Creation of a means for self-assessment;
• Replication of an award system by hundreds of organizations, including states, cities, companies, and not-for-profit organizations; and
• Creation of training programs.

            Throughout the life of the Baldrige Award Program, the principal uses of the Criteria have been for such other purposes. To date, more than one million copies of the Criteria have been disseminated, and a like number of copies have been duplicated by others. This compares with a total of 546 applicants for the Baldrige Award.
            Since the Baldrige Award was established in 1987, there have been 22 Award recipients (1988-1994). Award recipients have demonstrated a wide range of improvements and achievements, including product and service quality, productivity growth, customer satisfaction, reduced operating costs, and improved responsiveness.  Also, Award recipients are among the Nation's leaders in investment in developing the skills of the work force.
            Working Toward an Education Category Since the inception of the Baldrige Award in 1987, some educators have been involved in the Program through their service on the Award's Board of Examiners.  In addition, the Award recipients have sought to involve educators and educational organizations, locally and nationally. Also, some state and local award programs already include education categories.  In parallel with these Award developments, many educators have launched quality improvement efforts.  National initiatives such as Goals 2000 reflect a growing national consensus to strengthen education.  As a result of these and related developments, interest has grown in establishing a Baldrige Award category for education. In 1993, a decision was reached to launch Pilot activities in 1994 and 1995 to address the many issues that arise in extending eligibility to education.
 
Perceived Quality Assessment Instrumentation

            Parallel with the development of the Baldrige Award criteria, instrumentation was under development to apply principles of the system to education.  In 1993, an attempt was undertaken to develop instrumentation that focused on the factors of quality improvement in connection with school system organizational performance.  The Perceived Quality Assessment Instrument was developed for use with school districts by William K. Poston Jr. and Rashid M. Bax at Iowa State University in 1993-94.
            The instrument was built along the lines of the Baldrige criteria framework, and included seven dimensions:  leadership, information and analysis, strategic quality planning, human resource development and management, management of process quality, quality and operational results, and client focus and satisfaction.  Six to eight items were developed and included in each category.  Statements were developed addressing the operations and functions of school districts, and respondents were requested to report their perception of the current situation of their system on a five point Likert-type scale.  The respondents were also requested to indicate their perception of the ideal or preferred situation on the same scale on each statement.  Responses included "strongly agree, agree, neutral, disagree, and strongly disagree."

Calculating the Quality Effectiveness Index

            Unique to this instrument was a quality effectiveness index, which was a calculated ratio of the current situation and the ideal situation scores on each item, each of the seven dimensions, and for the total instrument.  The formula for the index was expressed as follows:

QEI = R_c / R_i  

QEI = Quality Effectiveness Index  Rc = Rating of Current Situation  Ri = Rating of Ideal or Preferred Situation

 

            Calculating the quality effectiveness index in this manner permitted a basis for comparison across school systems as to the level of quality considering the ideal or favored status of the system.  The expressed ratio expressed a bi-directional assessment of quality, and indicated a diminution of quality as the ratio decreased from 1.0, and an achievement of expectations as the ratio increased above 1.0.  In other words, if a school system had a quality effectiveness ratio of .50, it was only approximately 50% of the way toward the level of quality it perceived as desirable.  A quality effectiveness ratio of 1.25 indicates surpassing those same expectations by approximately 25% .
            Such an instrument in educational research can be very valuable if carefully planned and developed (Borg and Gail, 1989).  Self-assessment tools like the PQAI have been described in the professional literature as tools for obtaining an aerial view of the territory to be explored (Neuroth, Plastik, and Cleveland, 1992).  By plotting a district's current location on the map, educational leaders can determine what courses of action would be appropriate for improvement.
            Validation and reliability determinations for the instrument are discussed in the findings section of this report.  The Perceived Quality Assessment Instrument criteria are most explicit in the areas of organizational functioning pertaining to teaching and learning.  The focus on teaching and learning depends upon leadership and organizational expectations, information and analysis of assessment data for improvement, strategic planning for quality improvement and mission accomplishment, development and management of human resources congruent with participatory management, employment of process quality principles and procedures, monitoring of quality and operational results, and a clear focus on clientele and their satisfaction.
            The point values and Likert-type scale represent an initial attempt to provide a basis for scoring participating school systems in terms of progress in performance improvement.
 

Methodology of the Studies

Population of the Study

            The study involved school districts in the State of Iowa, a mid-western state with an agricultural-industrial economic climate and a rural-urban mix of communities.  The specific population of the study was identified prior to the commencement of the study, so that questions could be postulated toward the respondents selected (Borg & Gall, 1989).  A graduate seminar group of Ph.D. students and the major professor, common to all the students, was formed for the purpose of framing and jointly planning the study.  The seminar determined several aspects of the study, including the population selected, the variables measured and scrutinized, and the activities pursuant to the completion of the research on perceived quality improvement.
            Forty-four of Iowa's 360 districts were identified and selected to participate in the study by the graduate seminar group.  Participation was voluntary on the part of each school organization, and efforts were made to include systems representative of different sizes and configurations.  The districts selected were representative of the state as a whole in size and geographic distribution.  Agreements to cooperate in the study were received from each district superintendent in order to increase the response rate.  Instruments were delivered to the superintendents' offices for distribution to respondents.  The superintendents were also asked to have the system's official legal representative, commonly referred to as board secretaries, select the respondents randomly and distribute the surveys.
            A sample of personnel completed the study instrumentation.  The Perceived Quality Assessment Instrument (PQAI) was completed by the superintendent, all board members, two administrators, five teachers, three support personnel, and two high school students for a total of 720 possible respondents in the 44 districts. The other instruments were distributed in similar manner, with different personnel responding.
            Staff development information was elicited by a questionnaire designed specifically for that purpose, as was the information on compensation structures.  Both the Staff Development Questionnaire and the Executive Compensation Instrument were designed specifically for this study, and both were validated with standard review and developmental procedures.  Financial information used in the study was obtained from official published sources, provided by the Iowa State Department of Education.

Research Design and Variables of the Study

            A survey design was used to answer the research questions. The first instrument (PQAI) consisted of two parts: Part I - Demographic Information, and Part II Rating of School System Quality Components. The other instruments were drawn from research literature. The dependent variables were the ratios (Quality Effectiveness Index) between current and ideal perceptions of quality management in each district.  The specific calculation involved a ratio of the current perceived quality improvement status divided by the perceived ideal quality improvement rating.  The ratios also included the seven sub-scales: Leadership, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management, Management of Process Quality, Quality and Operational Results, Client Focus and Satisfaction. The independent variable of the study was the perception of quality of the districts' staff development programs. The sub-scale areas and demographic variables of the PQAI are shown in Table 1 below.
 

Table 1.  Sub-scale areas and demographic variables of the PQAI

Quality Components	Demographic Information
Leadership	Position
Information and Analysis	Home Annual Income
Strategic Quality Planning	Gender
Human Resource Development and Management	Age
Management of Process Quality	Level of Education
Quality and Operational Results	Years Experience in Job
Client Focus and Satisfaction

 

Development of the Instruments and Data Source

            The Perceived Quality Assessment Instrument (PQAI).  Use of the questionnaire in educational research can be very valuable if carefully planned and developed (Borg & Gall, 1989).  In this study, three questionnaire surveys were developed for use in this study, and a fourth data-gathering technique was also used.  The first survey instrument was based upon the Baldrige award criteria and labeled, "School System Perceived Quality Assessment Instrument" (Poston & Bax, 1994, see Appendix A).  The instrument was based upon the seven dimensions of the Malcolm Baldrige Award areas:  Leadership, Information and Analysis, Strategic Quality Planning, Human Resource Development and Management, Management of Process Quality, Quality and Operational Results, and Client Focus and Satisfaction.
            Six to eight items are included in each category with statements addressing the operations and policies of school districts. Respondents were asked to judge their current situation and the desired or ideal situation in their school system for each item using a Likert scale (Borg & Gall, 1989) of five possible responses for each (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree).  Demographic items were also included (position, income, gender, age, level of education, and years of experience in current or similar job) to aid in possible statistical comparisons and analyses of the groups.  The instrument was refined and improved following Bax's study (Bax, 1994).
            The Perceived Quality Assessment Instrument (PQAI) was developed and based upon principles of sound informational and research principles.  Borg and Gall (1989) report that specific behaviors can be predicted from attitude measures about those behaviors, therefore a Likert-type scale of five possible responses for each item was again used: Almost Never, Occasionally, Don't Know, Frequently, Almost Always (Knudsen, 1993).  The third response, Don't Know, was included in the third position to encourage either a negative or positive answer from respondents. Questions posed in closed form aid in the efficient quantification and analysis of results (Borg & Gall, 1989).  Some items were reversed from the others, so that not all statements would be written in similar (positive) format (Borg & Gall, 1989), and (2) so that respondents would not feel that every answer should be marked Almost Always to be "correct."  The length of the items and the length of the questionnaire itself were kept as short as possible for ease in understanding and increased chance of the instruments being returned (Borg & Gall, 1989).  In addition, several devices were used to encourage timely participation, including a self-addressed postage-paid mailer, phone calls, and personal contacts.
            The PQAI was validated by thirteen experts in quality improvement who were educational leaders, superintendents, university professors, or university officials with an understanding of quality processes.  The panel is listed in the Appendix.  Each member of the panel was asked to assure that each item accurately reflects the concepts purported to be measured by the instruments, to evaluate the items for clarity and completeness, and to make suggestions for improvement. Based on the recommendations of the panels, the instruments were revised.  The Perceived Quality Assessment Instrument was further pilot tested by a group of Iowa State University doctoral students studying quality management in public schools.  The PQAI instrument took approximately ten to fifteen minutes to complete.
 

Development of the Other Instruments and Data Sources

            Staff Development Instrumentation.  The District Staff Developmental Questionnaire (Johnson, 1995) was developed by one of the members of the study group (Johnson, 1995)  after reviewing staff development literature in the following areas:  (1) planning, (2) administrative support, (3) delivery, (4) follow-up, and (5) evaluation.  (Wood et al., 1982; Fullan, 1985; Joyce & Showers, 1988; Sparks and Loucks-Horsley, 1989; Senge, 1990; Rebore, 1991; Guskey, 1994; McBride et al., 1994; Sparks & Vaughn, 1994) Orlich, 1983; Guskey & Sparks, 1991; Templeman & Peters, 1992; Showers, 1985; Wade, 1985; Bates & Stachowski, 1991; Wood & Thompson, 1993).  The instrument was validated by ten staff development professionals recognized throughout the state and nation as very knowledgeable in this field.  A Likert scale approach was used for the instrument.

            Executive Compensation Instrumentation.  The Current and Preferred Methods of Awarding Superintendent Salary Increases Questionnaire (Behounek, 1996) survey instrument was developed following a review of related literature and an interview of ten superintendents and twelve board members about methods of awarding compensation to superintendents.  Twelve criteria were incorporated in the instrument, with five of the criteria based upon the superintendent's performance and seven of the criteria nonperformance related.  The instrument was validated by a panel of nine practicing administrators and researchers, and field tested with twelve board members.  A Likert-type scale was used for the instrument.

            Financial Information Instrumentation and Data Sources.  The source of financial information for the selected Iowa school districts was obtained from the official financial audits for each system from the 1992-93 school year, the most recent data available (Kirchoff, 1996).  The source of the audits used in the study was the State Auditor's Office, a division of government in Iowa.  The audits were used as the source of student enrollment, fund balances, revenues and expenditures.
 
Human Subjects Committee Approval

            The Iowa State University Committee on the Use of Human Subjects in Research reviewed all research projects as a part of this study and concluded that the rights and welfare of the human subjects were adequately protected, that risks were outweighed by the potential benefits and expected value of the knowledge sought, that confidentiality of data was assured, and that informed consent was obtained by appropriate procedures.
            All instruments employed the use of anonymity for respondents, as there was no request for names of respondents. However, the questionnaires were coded to identify which school system the respondents represented.  This was necessary for follow-up and data analysis.  The completion of the questionnaires was voluntary and constituted consent to participate in the research project. All questionnaires were kept secure throughout the duration of the study and filed for safekeeping after the study was concluded.

Data Collection Procedures

            Study instrumentation was personally delivered to each superintendent or his/her designee in each district in late February and early March of 1994. The accompanying directions for completing the surveys were self-explanatory.  Instructions were printed on the fronts and backs of the questionnaires to mail the completed instruments directly to Iowa State University by dropping them in the U.S. mail.  A postal permit was printed on the back of each. To increase the rate of return, phone calls were made to participating superintendents in April asking them to encourage those who had not yet responded to do so.
            As a result, 471 of the 720 PQAI surveys were received for a return rate of 65.4%.

Statistical Analysis of The Data

            Perceived Quality Assessment Instrument.  After the surveys were returned, the responses were entered into a Microsoft Excel spreadsheet program.  Only one of the PQAI instruments was discarded because it was damaged in the mail.  Four of the districts were removed from the study because an insufficient number of PQAI surveys were returned (one survey and was returned from two districts and three surveys were returned from two others).  As a result, 462 surveys representing forty districts were used for data analysis. The items were ranked on a scale ranging from 1 (strongly disagree) to 5 (strongly agree).  Means for each respondent and each district for each question were computed.  On the Perceived Quality Assessment Instrument, means for each respondent and district for the current and ideal situations were calculated as well as the ratio (quality effectiveness index) of the current divided by the ideal.
            Descriptive statistics were computed on the demographic variables of the PQAI.  Preliminary analyses using the one-way analysis of variance (ANOVA) were conducted to compare the responses of demographic groups.  Factor analysis was conducted on the PQAI to determine reliability.  The ANOVA and Scheffe' post-hoc method were used to determine differences between the four groups.
            Other instruments.  Spearman rho and Pearson product-moment correlation coefficients were computed to test for a relationship between responses on the PQAI and the staff development and compensation instruments and between financial factors and the PQAI.  Stepwise multiple regression was used to find a relationship between staff development and the seven PQAI dimensions.  The differences between high and low districts on the compensation and staff development instruments and financial factors were determined using the t test and analysis of variance techniques.
            The Cronbach alpha reliability coefficient was also calculated for the Staff Development Questionnaire and the Executive Compensation Instrument to determine the internal consistency of the total instruments.
 

Findings of the Studies

Introduction

            The findings of the study are organized into the following sections: (1) General Characteristics of the Sample, (2) Reliability Analysis of the Instruments, (3) Quality Management Perceptions, (4) Staff Development Perceptions, (5) Results of Hypotheses Tested, (6) Evaluation of the Perceived Quality Assessment Instrument, and (7) Summary.

General Characteristics of the Sample

            The primary purpose of this section is to describe the participants in this study who completed the Perceived Quality Assessment Instrument (PQAI) with respect to the following demographic variables: (a) position, (b) home annual income, (c) gender, (d) age, (e) level of education, and (f) years experience in current/similar job. The descriptive information is presented in Table 2 which follows.
 

Position - Of the 458 respondents who filled in this question, 132 (28.8%) were teachers, 76 support staff (16.6%), 68 administrators (14.8%), 30 superintendents (6.6%), 108 board members (23.6%), and 44 students (9.6%). Four people did not complete this category.

Home Annual Income - Of the 358 people answering this question, 24 reported earnings of less than $10,000 (6.7%), 77 from $10,000 to $29,999 (21.4%), 147 from $30,000 to 49,999 (40.9%), and 110 reported earnings of $50,000 or more (30.9%). There were 104 respondents who did not fill in this information.

Gender - The number of male and female participants reporting their gender was fairly even. Male respondents numbered 215 (52.6%) while females totaled 194 (47.4%). The number who did not fill in this item was 53.

Age - The age of the respondents was divided into five categories: (1) under 18, (2) 18-29, (3) 30-55, (4) 56-70, and (5) over 70. The largest group was the 3055 year category (73.8%). The second largest group was the 56-70 year group (11.9%). The smallest group was the over 70 category with only one person. Forty-two people did not report their age group.

Level of Education - The educational level of the respondents was classified into four categories: (1) less than a B.A. degree, (2) B.A. degree, (3) Master's degree, and (4) Doctorate degree. The largest group (151 people) represented those who had less than a B.A. degree, (34.6%). Those having a B.A. degree were the second largest group with 142 people (32.5%). The third largest group, those having a Master's degree, were very close with 129 people ((29.5%). Fifteen respondents (3.4%) reported having a Doctorate degree. Twenty-five respondents did not report their level of education.

Years of Experience - The years of experience of the respondents were divided into four categories: (1) under 5 years, (2) 5-10 years, (3) 11-25 years, and (4) 25 years or more. The findings revealed that 191 (44.3%) of the participants had 11-25 years of experience, 96 (22.3%) had 5-10 years of experience, 88 (20.4%) had under five years, and 56 (13%) had 25 or more years of experience. Thirty-one respondents did not complete this item.

 

Table 2.  Demographic information of respondents on the PQAI

Category		Frequency	Percent
Position	Teacher  Support Staff  Administrator  Superintendent  Board Member  Other  Total	132 76 68 30 108 44 458	28.8 16.6 14.8 6.6 23.6 9.6 100.0
Income	< $10,000  $10,000 - $29,999  $30,000 - $49,999  $50,000 +  Total	24 77 147 110 358	6.7 21.4 40.9 30.9 100.0
Gender	Male  Female  Total	215 194 409	52.6 47.4 100.0
Age	< 18  18 - 29  30 - 55  56 - 70  70 +  Total	23 36 310 50 1 420	5.5 8.6 73.8 11.9 0.2 100.0
Education	< Baccalaureate  Baccalaureate  Master's Degree  Doctorate  Total	151 142 129 15 437	34.6 32.5 29.5 3.4 100.0
Experience	< 5 years  5 - 10 years  11 -25 years  26 + years  Total	88 96 191 56 431	20.4 22.3 44.3 13.0 100.0

 

            One-way analysis of variance and t-test procedures were used to determine differences in the quality effectiveness index among different demographic groups. It was found that there were significant differences in responses by position, gender and education. The Scheff© method was used to analyze these differences and established that there were significant differences in the position category between the responses of teachers and administrators and between teachers and board members, both at the .05 level. A description of respondents by position follows.

Analysis by position

            In order to gain a better understanding of the respondents on the PQAI and their differing characteristics, further analysis was conducted on the position category by generating cross-tabulations with the other variables--income, gender, age, education and experience. The results are shown in Table 3 that follows.

             Position with income - As shown in the following Table , the majority of teachers (50) reported incomes in the $30,000 to $49,999 category with the second highest number (44) being at the over $50,000 level. Support staff reported 29 people in the $30,000 to $49,999 and 27 at the $10,000 to $29,999 level.
            For administrators, the two highest categories were over $50,000 (39) and $30,000 to $49,999 (28). Only one administrator reported making less than $29,999. All of the superintendents except two (27) reported incomes over $50,000, while the remaining two reported earnings in the $30,000 to $49,999 category. The majority of board members (51) were in the over $50,000 category, with 31 at the $30,000 to $49,999 level, and 16 at the $10,000 to $29,999 level. Of those in the other category (students), 23 reported earnings under $10,000 with the remainder (16) dispersed throughout the other three levels.
 
 

Table 3.  Demographic information of PQAI respondents by position

Category	Teacher	Support Staff	Admin.	Supt.	Board	Other	Total	Percent
Income (35 missing)    < $10,000    $10,000 - $29,999    $30,000 - $49,000    $50,000 +  Total	0 30 50 44 124	1 27 29 12 69	0 1 28 39 68	0 0 2 27 29	0 16 31 51 98	23 3 7 6 39	24  77  147  179  427	5.6 18.0 34.4 41.9 100.0
Gender (53 missing)      Male      Female    Total	39 82 121	21 48 69	50 10 60	22 0 22	65 30 95	18 24 42	215 194 409	52.6 47.4 100.0
Age (42 missing)      < 18      18 - 29      30 - 55      56 - 70      70 +    Total	1  18  93  10  0 122	1 4 56 9 0 70	0 1 54 8 0 63	0 1 22 6 0 29	0 0 80 14 1 95	21 12 5 3 0 41	23 36 310 50 1 420	5.5 8.6 73.8 11.9 .2 100.0
Education (25 missing)     < Baccalaureate     Baccalaureate     Masters Degree     Doctorate   Total	1 95 0 0 126	57 10 0 0 71	1 3 0 0 68	0 2 9 9 29	55 32 6 6 403	37 0 0 0 40	151 142 15 15 437	34.6 32.5 3.4 3.4 100.0
Experience (31 missing)     < 5 years     5 - 10 years     11 - 25 years     26 + years   Total	12 30 67 17 126	13 21 33 6 73	18 17 24 9 68	4 8 15 2 29	17 20 48 18 103	24 0 4 4 32	88 96 191 56 431	20.4 22.3 44.3 13.0 100.0

 

             Position with gender - There were over twice as many females as males in the teacher (82 and 39) and support staff (48 and 21) categories, as shown on Table 3.
            However, male administrators (50) outnumbered females (10) 5 to 1, while there were no female superintendents and 22 male superintendents who reported their gender. There were more than twice as many male board members (65) as females (30). In the other category, 18 were males and 24 were females.
            Position with age - The majority of the teachers (93) were in the 30-55 years age group, with 18 falling in the 18-29 category, 10 in the 56-70 division, and one reporting being under 18. Most of the support staff (56) reported being 30-55, with 9 in the 56-70 age group, 4 in the 30-55 range, and one under 18. Administrators, too, reported the majority (54) in the 30-55 group, with 8 in the 56-70 category, and one in the 18-29 age group. Twenty-two superintendents reported being in the 30-55 age range, with six in the 56-70 category, and only one in the 18-29 age group. No board members reported being under 30, while the vast majority (80) were in the 30-55 range, 14 in the 56-70 category, and 1 over 70. In the other group, 21 reported being under 18, 12 said they were 18-29, 5 filled in the 30-55 group, and 3 marked the 5670 age group.
            Position with education - Most of the teachers (95) reported having B.A. degrees, with 30 having Master's degrees, and one reporting less than a B.A. The majority of support staff (57) had less than a B.A. degree, with 10 having a B.A., and 4 having a master's degree. Most of the administrators (62) had a master's degree, with only 3 listing a B.A., and one filling in the less than B.A. category. Nearly two-thirds of the administrators (18) had master's degrees, while nine had doctorates, and two had completed their B.A.'s. Of the board members reporting, over half (55) had less than a B.A., while 32 had a B.A., 10 had a master's degree, and six held doctorates. In the other group, 37 reported having less than a B.A., while three filled in the master's degree response. It would appear that there is an inverse relationship between the education and income levels of the board members. While the majority of board members did not have a college degree, over half of them reported incomes of more than $50,000.
            Position with experience - The results from Table 2 show that the largest number of respondents in every group except other had 11-25 years experience in the current or similar job. The second highest level in every group except administrators and other was the 5-10 year category. The remainder of respondents were distributed fairly evenly between the other two categories.
 

Analysis of other demographics

            The ANOVA procedure revealed no significant differences on the quality effectiveness index in income levels, age, or experience. However, differences were found for education levels. Using Scheff©, significant differences were found between those with less than a B.A. and those with a B.A. degree. No other significant differences were found between the other education levels.
            Although the t test indicated a significant difference in the quality effectiveness index between males and females, when t tests were conducted for gender by position, no significant differences were found in any position level for gender. Thus the difference was only present when all levels were considered as a whole.

Reliability Analysis of the Instruments

            The SPSS program was utilized to determine the reliability of the Perceived Quality Assessment Instrument. Analyses were conducted for the current and ideal sections and for the overall scale (total instrument).  The alpha reliability coefficients are reported in the Table which follows. The alpha coefficients ranged from .68 to .85 for the current section of the instrument, with an overall reliability of .96.  The alpha coefficients were somewhat different for the ideal section, ranging from .61 to .89, with an overall reliability of .94, indicating a high positive correlation among all items.
            Table 4 below presents the information and reliability coefficients for the PQAI Instrument.
 
 

 Table 4.  Reliability analysis of current and ideal sections of the PQAI

	Reliability
PQAI Dimension	Item Numbers	Current	N	Ideal	N
Leadership	1 - 6	.84	454	.82	451
Information and Analysis	7 - 12	.85	443	.82	430
Strategic Quality Planning	13 - 18	.84	451	.87	445
H.R. Development & Mgmt.	19 - 25	.83	451	.61	441
Mgmt. of process Quality	26 - 31	.79	440	.85	437
Quality & Operational Results	32 - 37	.68	454	.75	448
Client Focus & Satisfaction	38 - 45	.85	453	.89	444
Overall	1 - 45	.96		.94

 
             The Cronbach alpha reliability coefficient was also calculated for the Staff Development Questionnaire to determine the internal consistency of the total instrument. Estimates of internal consistency are based on the average correlation among items within a test or instrument. The reliability coefficient for all thirty items was .95, again a high positive correlation.  This level of reliability was determined to be more than sufficient for the purposes of this research.
            On the Compensation Assessment Instrument, the Cronbach alpha reliability coefficient was also calculated to determine an estimate of the internal consistency of the instrument.  Reliability coefficients were calculated for the two sets of questions -- one for current practices and the other for ideal (or preferred) practices in compensating superintendents, or school system chief executive officers.  Table 5 below contains the reliability coefficients for the four sub-areas and the total instrument (Behounek, 1996).
 
 

Table 5.  Cronbach Alpha reliability coefficients for sub-areas of the Compensation Assessment Instrument 

Variable	Alpha	Number of Items
Current Practice
Non-performance related items	.75	7
Performance related items	.83	5
Ideal (Preferred) Practice
Non-performance related items	.87	7
Performance related items	.67	5
Total of all items	.67	24

 

            The level of reliability of the Compensation Assessment Instrument was determined to be adequate, albeit modest, for the purposes of this research (Nunnally and Durham, 1975).

Quality Management Perceptions

            The study sought to determine the perceived current and ideal levels of quality management in each participating district and the Quality Effectiveness Index ratio between the two assessments of perception.  Means were first calculated for the responses for each item for each district. The means of all of the current and ideal responses for each district were then figured. Current means ranged from 2.80 (district HH) to 3.92 (district MM).  Ideal means ranged from 3.77 (LL) to 4.56 (P and R).
            Finally, the ratio between the current and ideal means for each district were determined. The quality effectiveness index ranged from .6532 (district HH) to .9506 (district A). Four districts were removed from the study because of their low return (N for districts F and V = 1 survey each; N for districts I and M = 3 surveys each). It was felt that such a small sample from those districts would not give a reliable representation of the perceptions of the entire district.  The presentation in Table 6 below shows the distribution of means for current and ideal situations and the ratios for the remaining 40 districts.
 

Table 6.  Current and ideal means and quality effectiveness index for districts on the Perceived Quality Assessment Instrument 

District Code	Rank	Current Situation Mean	Ideal Situation Mean	Quality Effectiveness Index
A	1	3.60	3.79	.9506
AA	23	3.10	4.00	.7747
B	10	3.52	4.18	.8408
BB	33	3.44	4.66	.7371
C	7	3.70	4.32	.8572
CC	31	3.32	4.53	.7322
D	36	3.10	4.36	.7119
DD	15	3.39	4.23	.8005
E	30	3.08	4.15	.7419
EE	32	2.97	4.01	.7399
FF	4	3.66	4.15	.8859
G	21	3.44	4.37	.7865
GG	16	3.30	4.12	.8001
H	14	3.24	4.04	.8020
HH	40	2.81	4.30	.6532
II	28	3.22	4.29	.7500
J	13	3.45	4.26	.8083
JJ	11	3.56	4.31	.8250
K	19	3.33	4.19	.7943
KK	38	2.97	4.30	.6899
L	27	3.26	4.32	.7554
LL	5	3.50	3.95	.8843
MM	6	3.92	4.46	.8784
N	26	3.43	4.50	.7619
NN	22	3.40	4.42	.7703
O	3	3.77	4.23	.8909
OO	2	3.71	4.08	.9107
P	39	2.99	4.56	.6545
PP	8	3.64	4.29	.8501
Q	35	3.20	4.36	.7357
QQ	18	3.58	4.47	.7989
R	29	3.42	4.58	.7483
RR	20	3.56	4.50	.7889
S	25	3.23	4.23	.7628
T	24	3.26	4.25	.7669
U	9	3.46	4.09	.8480
W	34	3.27	4.44	.7369
X	17	3.44	4.31	.7990
Y	37	3.38	4.28	.7097
Z	12	3.65	4.44	.8234

 

Staff Development Perceptions and PQAI

            To determine the perceptions of teachers in the participating school districts about their systems staff development programs, means for each question for each district were calculated. Then the mean of all the responses for each district was determined. These means ranged from 1.963 to 3.789.  The same four districts (F, I, M, and V) were removed from the list for comparisons with the group used with the PQAI.  Ranks of districts on the Staff Development Questionnaire were compared with the districts' ranks on the PQAI, as shown in Table 7 below.

Table 7.  Comparison of ranks on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument 

District Code	SDQ Rank	PQAI Rank	Sub-Group
A	1	1	1
K	2	19	1
PP	3	8	1
J	4	13	1
JJ	5	11	1
LL	6	5	1
QQ	7	18	1
L	8	27	1
C	9	7	1
II	10	28	1
O	11	3	2
Q	12	26	2
OO	13	2	2
DD	14	15	2
H	15	14	2
B	16	10	2
RR	17	20	2
P	18	39	2
R	19	29	2
MM	20	6	2
N	21	26	3
BB	22	26	3
Z	23	16	3
GG	24	16	3
NN	25	22	3
AA	26	23	3
W	27	34	3
U	28	9	3
D	29	36	3
T	30	24	3
X	31	17	4
S	32	25	4
Y	33	37	4
EE	34	32	4
E	35	30	4
CC	36	31	4
G	37	21	4
FF	38	4	4
HH	39	40	4
KK	40	38	4

            The forty districts, listed above, were divided into four groups for further analysis.  The sub-groups were comprised of groups of ten districts each, according to their rank on the Staff Development Questionnaire.  A one-way analysis of variance was calculated to determine differences in means of the four sub-groups.  The results of the ANOVA are shown below in Table 8 below.
 
 

Table 8.  One-way analysis of variance of group means on the Staff Development Questionnaire 

Source	df	SS	MS	F	Fcv
Between groups	3	7.53	2.51	129.53	4.51
Within groups	36	.70	.02
Total	39	8.22

                            p= <.0001

             The four groups appeared to differ in terms of their group means on the Staff Development Questionnaire, as illustrated in the Table (Table 9) below.
 
 

Table 9.  Sub-group means and standard deviations on the Staff Development Questionnaire 

Group	N	Mean	St.Dev.
1	10	3.36	.2129
2	10	2.93	.0720
3	10	2.65	.0859
4	10	2.17	.1399

 

             A one way analysis of variance was then employed to determine differences in means between and among the four sub-groups.  Significant differences were found among the groups, and a Scheff© method was used to determine which sub-groups differed from one another.  All groups were found to differ from each other significantly, as shown in Table 10 which follows:
 
 

Table 10.  Comparison of group means on the Staff Development Questionnaire 

Sub-Group Comparison	Mean Difference	Scheffe F test
1 vs. 2	.4366	16.4*
1 vs. 3	.7155	44.05*
1 vs. 4	1.1944	122.76*
2 vs. 3	.2789	6.69*
2 vs. 4	.7578	49.42*
3 vs. 4	.4789	19.74*

                                     *Significant at .01 level
 

PQAI and SDQ Relationships

            The relationship between the PQAI measurement of perceived quality and the Staff Development Questionnaire was explored.  The composite district ratings of staff development programs were correlated with the Quality Effectiveness Index computations.  A Spearman rho test was used to calculate correlations between the two indices.  A correlation of .50131 was found, and it was significant at the .01 level of confidence.  This relationship was determined to be moderate, but significant.
            A second exploration was conducted to determine if there were any relationships between ratings of perceived levels of staff development quality and the seven dimensions of district quality (current) measured by the PQAI in the participating districts.
            To accomplish this purpose, means of the current perceptions in each of the seven quality dimensions of the Perceived Quality Assessment Instrument for the ten highest ranking districts on the Staff Development Questionnaire were computed.  The mean of each district on the Staff Development Questionnaire was compared with the mean of each quality dimension for these districts.  Stepwise multiple regression was used to determine the relationship of the measured quality of staff development to the seven quality dimensions. The seven variables were entered one at a time and a significance test was conducted to determine the contribution of each (Hinkle, Wiersma, & Jurs, 1988). The stepwise solution was terminated when the remaining variables did not make a statistically significant contribution to the regression.
            It was found that three of the seven dimensions--Client Focus and Satisfaction, Quality and Operational Results, and Management of Process Quality--were good predictors of levels of quality staff development. The adjusted R2 (squared) was .956 indicating that 96 percent of the variance on the Staff Development Questionnaire was explained by these three dimensions.  The remaining four dimensions -- Leadership, Information and Analysis, Strategic Quality Planning, and Human Resource Development and Management -- did not predict levels of district staff development in any significant degree.
            Correlation matrices were also constructed in order to show the interrelationships between all the sub-scales on the Perceived Quality Assessment Instrument. Table 11 shows the correlation coefficients for each of the seven sub-scales. As a rule of thumb, correlation coefficients between 00 and 30 show little if any correlation; 30 to 50, a low correlation; 50 to 70, a moderate correlation; 70 to 90, a high correlation; and .90 to 1.00, a very high correlation (Hinkle, Wiersma, & Jurs, 1988).
            Using these guidelines, the matrix shows that the majority of the correlations are moderate positive correlations. None of the relationships are below 50, with the lowest being between Leadership and Quality and Operational Results (.52).
            Five of the relationships were in the high positive category: Human Resource Development and Management and Strategic Quality Planning (.77), Human Resource Development and Management of Process Quality (.73), Management of Process Quality and Quality and Operational Results (.71), Client Focus and Satisfaction and Management of Process Quality (.72), and Client Focus and Satisfaction and Quality and Operational Results (.74). These relationships show that all seven dimensions are related to each other and measure the same concept -- district quality management.
 
 
 

Table 11. Correlation matrix for the seven dimensions of the PQAI

PQAI Sub-scale	Leader-  ship	Infor. &  Analysis	Strategic  Planning	H.R. Dev.  & Mgt.	Process   Quality	Quality  Results	Client  Focus
Leadership	1.00
Information	.63	1.00
Planning	.61	.69	1.00
H.R. Mgt.	.62	.67	.77	1.00
Process	.63	.69	.67	.73	1.00
Results	.52	.62	.62	.67	.71	1.00
Focus	.60	.64	.64	.69	.72	.74	1.00

 

            Still another exploration was conducted into the area of determining whether or not districts ranked highest or lowest in perceived staff development quality differed significantly when measured by the perceived quality effectiveness index.
            Table 12 shows these highest and lowest districts, their ranks on both instruments and their mean scores for each.
 

Table 12.  Mean scores and ranks for the highest and lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument
 

Table 12.  Mean scores and ranks for the highest and lowest ten districts on the Staff Development Questionnaire and the Perceived Quality Assessment Instrument

District	Rank on SDQ	Group	SDQ Mean	PQAI Mean	Rank on PQAI
A	1	1	3.789	.9506	1
K	2	1	3.567	.7943	19
PP	3	1	4.442	.8501	8
J	4	1	3.427	.8083	13
JJ	5	1	3.408	.8250	11
LL	6	1	3.278	.8843	5
CD	7	1	3.278	.7989	18
L	8	1	3.273	.7554	27
C	9	1	3.113	.8572	7
II	10	1	3.067	.7500	28
X	31	4	2.408	.7990	17
S	32	4	2.310	.7628	25
Y	33	4	2/280	.7097	37
EE	34	4	2.211	.7399	32
E	35	4	2.203	.7419	30
CC	36	4	2.167	.7322	31
G	37	4	2.150	.7865	21
P	38	4	2.042	.8859	4
HH	39	4	1.994	.6532	40
KK	40	4	1.963	.6899	38

 

            Using the unpaired t test on the means of the two groups on both instruments, significant differences were found for the highest and lowest groups on the Perceived Quality Assessment Instrument.  A correlation coefficient (Pearson product-moment) was then calculated among groups compared on both instruments. A moderate positive correlation (.564) was found for the top ten districts which was significant at the .05 level. The correlation between the bottom ten districts on both instruments (.264) was not significant. However, when district FF was removed which ranked high (number 4) on the Perceived Quality Assessment Instrument, the correlation rose to .710, a high positive correlation.  It was determined that the higher a system demonstrated perceived quality in staff development, the higher its perceived quality effectiveness index.

Analysis of Executive Compensation Perceptions and PQAI

            The compensation of executives and leaders has often been thought to be instrumental in influencing the level of  performance on the job.  In short, some believe that money can motivate (Poston and Frase, 1992).  Deming, long considered the "father of quality management," however, felt that the theory of psychology eroded any confidence that money could motivate individuals on the job (Deming, 1986).  His notion was that money and employee compensation had little to do with the quality of an organization.
            To test that theory in part, this study undertook to examine the perceptions of school board members and superintendents about the effect or influence of salary compensation upon the quality of an organization.  Perceptions from board members and superintendents were elicited with the instrument earlier described.  The following Table demonstrates the tabulated results of the perceptual assessment.
 
 

Table 13.  Aggregate responses of superintendents and board presidents concerning practices and preferences for salary increases

		Board Presidents			Superintendents
Item	Belief statement	N	Mean	St.Dev.	N	Mean	St.Dev.
CURRENT: Unrelated to Performance
1	Length of service to system	44	2.67	.94	44	2.68	.93
2	Increased formal education	44	2.75	.97	44	2.27	.95
3	Cost of living adjustment	44	2.86	1.00	44	2.48	1.09
4	Growth in revenues	44	3.46	1.15	44	3.50	1.07
5	Settlement of teacher contract	43	3.56	1.22	44	3.80	1.17
6	Regional colleagues status	44	3.39	.99	44	3.48	1.07
7	Student enrollment change	44	1.86	1.02	44	2.16	1.16
	Total	44	2.94	1.17	44	2.91	1.25
CURRENT: Related to Performance
8	Performance on job	44	3.96	1.16	44	3.73	.85
9	Achievement of set goals	44	3.61	1.13	44	3.11	.99
10	School Board evaluation	44	3.77	1.05	44	3.61	.99
11	Student achievement	44	2.77	1.14	44	2.02	1.02
12	Test score results	44	2.05	1.12	44	1.73	.90
	Total	44	3.23	1.32	44	2.84	1.25
IDEAL: Unrelated to Performance
1	Length of service to system	44	2.55	1.02	44	2.97	1.05
2	Increased formal education	44	3.41	.79	44	3.66	.78
3	Cost of living adjustment	44	3.41	.79	44	3.41	1.00
4	Growth in revenues	44	3.02	1.00	44	3.48	.95
5	Settlement of teacher contract	44	3.46	1.07	44	3.21	1.10
6	Regional colleagues status	44	3.07	1.13	44	4.18	.62
7	Student enrollment chnage	44	2.46	.79	44	2.59	.87
	Total	44	3.07	1.04	44	3.36	1.03
IDEAL: Related to Performance
8	Performance on job	44	4.36	.72	44	4.59	.50
9	Achievement of set goals	44	4.18	.79	44	4.32	.86
10	School Board evaluation	44	4.11	.75	44	4.43	.50
11	Student achievement	44	3.21	.85	44	2.84	1.08
12	Test score results	44	2.55	.82	44	2.02	.88
	Total	44	3.68	1.05	44	3.64	1.29

 

            For the purposes of this study, the strength of agreement indicated by each level of response was categorized as low if less than 2.25, moderate if between 2.26 and 3.75, and high if more than 3.76.  These designations were selected arbitrarily for purposes of comparison only.  The results in Table 13 represent superintendent and board president perceptions on how increases for superintendents are awarded in current practice and how they should be awarded given conditions of ideal practice.
            Board presidents and superintendents agreed that increases in enrollment have little to do with compensation, but levels of compensation awarded pursuant to teacher contract deliberations have a high impact upon the level of superintendent's compensation.  In the area of performance-related factors, the highest factor of influence to both board presidents and superintendents was the superintendent's performance on the job.  The lowest relationship to compensation was perceived to be with standardized test results.  Ironically, this measure of organizational effectiveness had low agreement as to its importance in determining compensation.
            When considering the ideal situation, board presidents and superintendents agreed that on-the-job performance, achievement of district goals, and board evaluation of the superintendent should have the greatest level of influence on the superintendent's compensation.  Ideally, as in the current status, student test performance was perceived of low importance.  In non-performance rated ideal conditions, superintendents favored most strongly the use of "benchmarking" salaries of peers or colleagues in the region for use in determining the salary of the superintendent.  Changes in enrollment fared no better in perceptions of ideal factors than with current conditions, as it was rated as the lowest in agreement for both groups.
            To more adequately compare board presidents' and superintendents' perceptions about compensation, the mean scores of both groups were subjected to analysis.  Unpaired, two-tailed t-tests were used to determine if significant differences existed between the two groups on the four areas of the study.  Responses from superintendents and board presidents were also compared between performance and non-performance factors and current practice and ideal situation.  In these cases, paired, two-tailed t-tests were utilized.  Significance was set at the .05 level of confidence for the purpose of this analysis.
 
 

Table 14.  t-Test results of group means for determining superintendent compensation

Group	N	St. Dev.	Mean	t	P
Current:  Unrelated to Performance
Board Presidents	44	.50	2.93	.18	.857
Superintendents	44	.71	2.91
Current:  Related to Performance
Board Presidents	44	.90	3.23	2.2	.030
Superintendents	44	.76	2.84
Ideal:  Unrelated to Performance
Board Presidents	44	.49	3.07	-2.70	.008
Superintendents	44	.51	3.36
Ideal:  Related to Performance
Board Presidents	44	.46	3.68	.40	.690
Superintendents	44	.50	3.64

 

            In the area of perceptions of board presidents and superintendents considering the importance of non-performance related job factors, the analysis revealed that there was no significant difference between the two groups.  On the other hand, when looking at how they two group means compare on performance-related job factors, there was a significant difference between board presidents and superintendents in their perceptions.  Board presidents felt more strongly about using performance in the determination of compensation for the chief executive.
            Two more tests of sign