MOD 02 DOC 670
Module 2 – Background
DESIGNING THE STUDY AND ASSESSMENT OF THE RESEARCH CONTEXT
Required Reading
Chapters 1 and 2 in:
Cooperrider, D. L., Whitney, D., & Stavros, J. M. (2008).Appreciative inquiry handbook: For leaders of change.Oakland, CA: Berrett-Koehler Publishing. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter36089in the search bar.
Preface in:
Dean, S., & Illowsky, B. (2014).Collaborative Statistics.Connexions: Rice University. Creative Commons License 3.0. Retrieved fromhttps://cnx.org/contents/[emailprotected]:[emailprotected]/Preface
Chapter 1 in:
Dean, S., & Illowsky, B. (2014).Collaborative Statistics.Connexions: Rice University. Creative Commons License 3.0. Retrieved fromhttps://cnx.org/contents/[emailprotected]:[emailprotected]/Video-Lecture-1-Sampling-and-Data
Chapter 1 in:
Goodyear, L., Barela, E., Jewiss, J., & Usinger, J. (2014).Qualitative inquiry in evaluation: From theory to practice.Jossey-Bass: Hoboken, NJ. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter80713in the search bar.
Chapters 3 and 4 in:
Ng, W., & Coakes, E. (2014).Business research: Enjoy creating, developing and writing your business project.Krogan Page: London, UK. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter58388in the search bar.
Chapter 1 in:
Phillips, P.P., Phillips, J.J., & Aaron, B. (2013).Survey basics.Association for Talent Development. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter53520in the search bar.
Stockberger, D. (2016).Introductory statistics: Concepts, models, and applications.Missouri State. Retrieved fromhttp://www.psychstat.missouristate.edu/introbook/sbk19.htm
Chapters 3, 4, and 14 in:
Swanson, R. A., & Holton, E. F. (2005).Research in organizations: Foundations and methods of inquiry.Berrett-Koehler Publishers: Oakland, CA. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter11859in the search bar.
Chapters 4 and 5 in:
Walker, S. (2012).Employee engagement & communication research: Measurement, strategy, & action.Krogan Page: London, UK. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter47012in the search bar.
Excel Resources
Brown, N., Lave, B., Romey, J., Schatz, M., & Shingledecker, M. (2018)Beginning Excel. OpenOregon, Creative Commons License. Retrieved fromhttps://openoregon.pressbooks.pub/beginningexcel/andhttps://openoregon.pressbooks.pub/beginningexcel/front-matter/introduction/
ExcellsFun. (2016, May 20).Highline Excel 2016 class 15: Excel charts to visualize data: Comprehensive lesson 11 chart examples[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=xLmtGk7Ymy8&t=2003s
Chapters 1 and 2 in:
Harvey, G. (2016).Excel 2016 for Dummies. John Wiley & Sons. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter117498in the search bar.
Book II: Chapters 14 and
Book V: Chapter 1 in:
Harvey, G. (2016).Excel 2016 All-in-One For Dummies. John Wiley & Sons. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter112925in the search bar.
Kaceli, S. (2016, January 24).Excel 2016 Tutorial: A comprehensive guide on Excel for anyone
[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=8lXerL3DHRw.Note:This video runs for 2 hours.
Video Material
BSC Designer. (2012, December 2).Best practice tips for creating key performance indicators[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=91SKwBX419k
ProfKLJ. (2009, January 28).Understanding variables & indicators[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=4Xc20ag_ckM
Daigle, D. (2014, June 5).Operationalization of concepts[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=36UZ9rKSa90
comresearch224. (2014, July 9). Conceptualization, operationalization, units of analysis, levels of measurement [Video file]. Retrieved fromhttps://www.youtube.com/watch?v=uaqUzUFHgUg
Eveland, J. D. (2017, October 15).Diagramming research models[Video file]. Retrieved fromhttps://www.youtube.com/watch?v=LevvMIug5fE&feature=youtu.be
Module 2 – Measures and Levels of Analysis
DESIGNING THE STUDY AND ASSESSMENT OF THE RESEARCH CONTEXT
One of the hallmarks of quality research is verisimilitudethat is, the appearance that the study really reflects the situation studied. This is the global equivalent of face validitythat most basic element of validity that says it makes sense. If it doesnt make any sense, then it really doesnt matter how sophisticated the modeling might be, or how elegant the sampling frame. Our judgments about verisimilitude are made primarily on the basis of our own experiences, augmented by training and reinforcement. They are generally forgiving judgments; that is, you have to be pretty dramatically off base to be noticed. But once you have lost your basic sense credibility, you never get it back, at least with that audience. Because of this, it is important to frame your own studies in a common-sense as well as research-based context.
So why multiple levels? What is there about organizational research that compels us to this fairly complicated kind of approach? The answer is, in a word, verisimilitude. Our own experiences of organizations, in which weve been embedded sometimes consciously and sometimes unconsciously for virtually our whole lifetimes, have taken place on many levels simultaneously. Even as we plan our individual careers and advancement, we know that we have to take into consideration the people with whom we work (group), the overall policies of the system (organization), and the social and economic environment (industry/society). To try to abstract one of these levels out and plan our career by that alone would be ruinous. The fact is, we can think quite easily about one level while holding the others in the back of our head or at worst in our scratch notes.
But formal research does not have the luxury of holding things back; it has to embrace everything in more or less the same fashion. Thus, when we are confronted with research that does not address the multilevel nature of organizational phenomena, it is at best vaguely unsatisfying and at worst credibility-destroying. This doesnt mean that every studyyours includedhas to implement the full rigors of a multilevel approach, but it does mean that in some way the research must acknowledge the complexities of organizational life and the degree to which different kinds of aggregations of people, systems, and tools condition and affect how we act and react.
So how do we as researchers move toward reflecting this multilevel organizational context? First, by being clear about what we mean by level. Different organizations are likely to employ different names for the levels within their hierarchies, but they all generally fall into categories based on size. For researchers, the key aspect is level of analysisa term simply referring to the units from which data are collected and their position relative to a generalized hierarchy, either official or imputed according to the size of the unit. For research purposes, we typically distinguish at least between the individual level of analysis, the group level, the organizational level, and the sector or industry level. Obviously, there are finer distinctions to be drawn and we often find ourselves in a position of having to draw them. That is, we may have data on individuals, gained through a survey, that we then aggregate to perhaps a workgroup level, then to a division level, into a plant level, before we reach the level of the organization. Levels of analysis pose problems that are both conceptual and methodological; each time we aggregate data to produce new composite variables, we introduce error, misattribution, and potentially even misrepresentation of the meaning of the data. In this module, we examine some of the problems posed when there are multiple levels of analysis in research and explore some of the potential solutions.
Make no mistake; you cannot escape issues of levels of analysis in your research. You will be asked about this question and you will be expected to be explicit about your analysis and why you believe that the level you have chosen is the appropriate level at which to form conclusions related to your research questions. It is in fact highly probable that you will need to use some variety of a multilevel model for at least part of your work, and these models are particularly difficult to specify and then to analyze. From this module, you will come toif not exactly a sense of security about levels of analysisat least an awareness that the problem exists and that there are ways around it. It is precisely the issues of levels of analysis that makes organizational research distinct from other kinds of work. Therefore, it is incumbent upon us to honor and embrace these problems, not try to sweep them away.
Robert McCallum, a professor of organizational psychology at Ohio State University, has neatly described the problem:
As a final major topic of comment, I wish to turn to the problem of level of analysis. This is of course a long-standing and difficult issue in the I-O field because many research questions involve individuals functioning in groups. Difficulties arise with regard to defining variables and levels at which they should be measured (e.g., climate, leadership), as well as in defining the level/s at which the research question or theory is to be investigated. A number of important papers in recent years (Klein, Dansereau, & Hall, 1994; House, Rousseau, & Thomas-Hunt, 1995; Rousseau, 1985) have emphasized the point that most problems studied in organizational research are inherently multilevel in nature and that researchers should take this into account at all stages of research, from theory development to data collection to data analysis. Katzell (1994) identifies the study of multilevel phenomena as a meta-trend in the I-O field.
When a problem is multilevel in nature, micro or macro views will lead to misspecification of theory by ignoring the multilevel nature of the phenomena and the relevance of variables at one level to variables at another level. For example, in studying individual job performance, there are undoubtedly both individual level variables (such as motivation and ability) and group level variables (such as climate and norms) that are relevant predictors. When a theory appropriately takes into account the multilevel nature of the problem, the gathering of appropriate data is facilitated. Units can be sampled at whatever levels are relevant (e.g., sampling organizations and sampling individuals within organizations), and variables can be measured at appropriate levels. Data can then be analyzed so as to take into account the multilevel structure of both the research questions and the data. There is considerable methodological literature about problems caused by aggregation and disaggregation of measures, thereby ignoring the multilevel structure of the data. Such procedures can yield severely biased results as well as invalid conclusions, such as the well-known ecological fallacy wherein one uses group-level data to draw conclusions about individuals.
Given the kinds of research questions that we ask in our dissertations in business, it is virtually inevitable that we will run up against the methodological difficulties referred to here. This is a classic double-bindour theories are in significant measure incompatible with the strictures of the research methods that we rely on to test them or otherwise accumulate evidence regarding their utility.
Organizational research, whether focused primarily at the individual level or the organizational (or higher) level, almost inevitably becomes entwined with constructs and variables that properly apply to other levels. Unless the methodological concerns are faced squarely and dealt with, analysis of these mixed-level models can lead to very misleading conclusions. The trick for side-stepping these problems is, unsurprisingly, careful specification of the model in question and application of some design and analysis approaches adapted to these situations. Easier said than done.
Attending to these concerns about levels of analysis serves at least two purposes. First, the issue is a serious problem in its own right, and a number of students have floundered over how to manage it. Second, and perhaps even more important, looking carefully at how researchers have thought about and then applied in the field some theories that are complex (many and multiple connections) even when they are simple (few variables), and managed ambiguous operationalization of constructs in creative ways, is a very good way to review the process of creating and applying what we have called the analytical framework or modelthat which distinguishes true scientific research from mere accumulation of data.
Individual level models of behavior in organizations are generally relatively simple in structure and in definition. That is why a lot of organizational research focuses on individuals. But when we decide to make the group or the entire organization our unit of analysis, we enter a different world. In the 1880s the U.S. Supreme Court decided that a corporation was legally a person and therefore entitled to the same rights to due process as anybody else; pretty much ever since then weve been explicitly or implicitly using the language of people to talk about organizations: Dow Chemical was a BAD boy!, The Department of Justice thinks, TUI has been growing so fast that its already gone through three pair of shoes, or something like that. All the Supreme Court did was codify the process weve been going through since prehistory (Gogs warriors are like a giant cave bear!)personifying the decisions of organized groups of people (Rome has decided) whether they are identified as decisions of individuals (It is the will of Caesar that), of groups acting in the name of many (The Inquisition sentences you), or of a person or persons unknown somewhere in the system (The President of the United States has decided that). Somehow organizations manage to feel like people to us and take on characteristics that function for both outsiders and insiders as a sort of personalitythe look and feel aspects that we sometimes term organizational culture.
Some people are strong personalities, some are bland; but all are equal in the sight of the law, and all are equal when handed a questionnaire, a Scantron form, and a No. 2 pencil or a webform. Likewise, whether an organization has a strong culture or not much at all, it is equally researchable at the organizational level of analysis. As we saw in relation to the individual level, certain problems are generally understood to be best addressed by considering the group or organization to be a single unit interacting with other like units and possessing characteristics of its own. These characteristics, which form the constructs and variables of organizational-level analysis, may be global (pertaining to the overall properties of the unit itself), shared (pertaining to properties of the units components that all possess equally), or configural (pertaining to properties of the units components that may vary across them). They may be measured directly (for global and some shared properties), or (for shared and configurable properties) by aggregating or averaging features of the components, or even by attributing to the unit characteristics of a larger unit of which it is a part. (In the background information for this module, youll find a presentation that reviews and expands upon these distinctions.)
Some of these characteristics mean much the same thing whether they are measured at the individual level (e.g., age) or at the collective level (e.g., the average age of group members). However, some exist only at the collective levelfor example, the age of the workgroupalso known as the time elapsed since its establishment as a distinct unit. The variables that are used to describe social networks are good examples of such characteristics; although they are often derived from data collected from individuals, the manner of their aggregation changes their interpretation. In a social network, for example, an individual has the characteristic of centrality, often expressed as the proportion of other members of the network s/he is in contact with. The overall network, on the other hand, has a characteristic called centralization reflecting the dependence of the overall connection structure on differing proportions of members; while it may be calculated from the centrality indices of its members, a centralization score has no direct meaning in reference to any of those members as such, but only in reference to the group as it is compared to other groups. Any topic that involves relationships among units, including the critical themes of power, conflict, and attraction, is likely to have such ambiguous variables that can be defined, with somewhat different meanings, at multiple levels of analysis. Recognizing that we are dealing with multi-level phenomena and that we need to make appropriate adjustments is not always easy. For example, power at the organizational level is rather different from, and more complicated than, power at the level of the individual; it is misleading to estimate that an organizational units source of power is simply the aggregate power of its individuals. Even the similarity of words can confuse; perhaps we should be talking aboutpoweriandpowero, or something like that.
The behavior of groups and organizations (holding off for the moment in addressing the fact that behavior on the part of an organization is analogous to but conceptually distinct from behavior on the part of individuals) is of interest to us in general because such behavior shapes the contexts within which we all live in critical waysphysically, socially, economically, and culturally. Students of business care about organizational behavior in particular because collective behavior is what our field is all aboutbusinessistransactions. Its all right to think of a one-person firm, but there are no one-firm economies. While models of individual-level behavior are important to the field, the behavior in question is generally behavior in relation to some organization or social setting. The core problems of business research are problems posed by, with, for, and to organizations.
We have called the place where we study such collectives the organizational level of analysis, but the same features actually describe the analysis of any aggregation of units beyond the individual, from the 2.5-person nuclear family to the entire family of humankind. The term organization lacks any precise meaning as to its size. In general, the appropriate unit is defined for researchers by the nature of the behavior we want to studythe unit of analysis is generally the smallest aggregate capable of undertaking such behavior as a unit. This might be a workgroup, a branch or division within the hierarchy of a larger organization, the organization itself (in its capacity as a legal person), or some larger conglomeration of organizations acting collectively (sectors, industries, nations, worlds [First through Fourth, at least], etc.) For example, if we want to conduct a study on corporate crime, we need to be at a level where there is something definitely corporate about whatever crime is being committed in terms of collective actors.
To complicate matters further, groups are almost always nested within other groups, just as individuals are nested within groups, and such nesting forms a critical part of the environment or context within which behavior takes place. However large or small, any aggregation of human beings is more complex and more difficult to understand and predict in some ways than an individual. Aggregates are also, somewhat paradoxically, simpler and easier to understand and predict, provided that we accept certain limits on what we can hope to comprehend. While it may be an impossibly complicated task to understand how an organization makes a decision, if we accept that each of these terms represents a sort of black box whose innards are incomprehensible but which produces recognizable and consistent output, it is then possible to create models of reasonably simple and parsimonious proportions, using these black box constructs as variables, that actually account for large parts of the variance of much organizational behavior, usually much more than can be accounted for in any individual-level research however many variables we choose to include. It is partly for this reason that group and organizational level studies make up the bulk of business studies.
The other main incentive for organizational-level research is what we might term the 800-pound gorilla effect. As we have noted, this is the immense impact that even small aspects of the behavior of large human aggregates can have for us all. If the fluttering of the wings of a butterfly in the Amazon can cause tornadoes in Kansas, as the chaos theorists would assert, then when IBM gets the organizational sniffles, an enormous number of people wind up sneezing or even with pneumonia, whether or not they have anything to do with IBM directly. There is a strong perception in many quarters that we cannot now even predict the actions of many organizations, let alone control them, and the complex social web within which chaos theory would see us as all embedded rather guarantees that our lack of even marginal understanding of many organizational phenomena will have more or less unpleasant consequences for substantial proportions of the population. Our very survival may well be bound up in our ability to control or at least predict how organizations, both large and small, will shape the world of today and of tomorrow.
Conducting research at the aggregate levelgroup, organization, industry, etc.introduces or compounds some problems such as sampling, identification of respondents, time horizons, research logistics, meaning and interpretation of data, and more. While many of the more interesting problems of business are properly addressed at this level. the practicalities of research often prohibit studying them directly, particularly for dissertation-scope projects. There are creative alternatives available, and the first step toward them is understanding that normal science prescriptions for research, primarily formulated in the simpler world of individual units, often need to be reinterpreted in the more complex environment of the organization.
Lets return our focus from the generic issues of research to the more particular world of your own research involvement and interests. The overall theme of this discourse, as we said at the beginning, has been twofold: first, to review a range of significant current research from a range of significant research publications and sources, and second, to do so in the context of some of the more intractable issues that face students in preparing their dissertation proposals. For this term, we have chosen to attend primarily to three of these issues: the close interdependence of research questions, theory, and research methods, the nature of quality research, and the choice and consequences of appropriate levels of analysis at which to address particular questions.
Module 2 – Case
DESIGNING THE STUDY AND ASSESSMENT OF THE RESEARCH CONTEXT
Assignment Overview
In this Case, you will be reading materials relating to definition of constructs, operational definition, and measurement. Then, you will be asked to create a list of such constructs and measures for your own study, taking into consideration the issues you have been reviewing and the decisions that you previously made regarding research designs. You will also be asked to identify control and/or environmental factors that you wont be able to include directly in your study, but which might affect your results.
As the course progresses, you will be making a series of decisions in which you will increasingly specify the structure of your project. These decisions build on each other in critical ways. However, you are not locked into previous decisions if you subsequently decide you need to change direction. You have an opportunity to go back and revise and/or extend the previous section completed in the previous module. If you do revise the earlier section, please include it and indicate what changes you have made to it. Case grades given for the first four modules will be advisory rather than final. Your ultimate grade will be based on the completed methodology at the end of the course.
Case Assignment
Prepare a 5- to 7-page paper in accordance with the following Assignment Expectations, describing the measurement issues for your project in accordance with the following outline:
List of main concepts in your study; why you believe that each of them is important.
Any explicit or implicit model that you may believe connects these concepts and that you propose to explore, if any. See above on creating and illustrating models.
Initial operational definitions of your study concepts, arranged in the form of this grid (please replace our illustration with your information):
1.
Concepts
Constructs
Variables/
Indicators
Measures
Source
Study or control
Level of analysis
Love
Love of children
Time spent with child
Minutes of direct contact per week
Observation
Study
Individual
Quality of interaction
Assessments immediately after interaction; qualitative ordinal scale 1-5 low-high
Questionnaire to participants
Study
Individual
Gender
Self-expressed gender
Gender category
Self-identification
Participant query
Control
Individual
Profitability
ROI
ROI
Calculated rate
Organizational records
Study
Organizational
1. List any concerns you may have about measurement or the general availability of the data you need.
Assignment Expectations
Length:The written component of this assignment should be 57 pages long (double-spaced) without counting the cover page and reference page.
Organization:Subheadings should be used to organize your paper according to the questions.
Grammar and Spelling:While no points are deducted for minor errors, assignments are expected to adhere to standard guidelines of grammar, spelling, punctuation, and sentence syntax. Points may be deducted if grammar and spelling impact clarity. We encourage you to use tools such asgrammarly.comand proofread your paper before submission.
When you write your paper make sure you do the following:
Answer the assignment questions directly.
Stay focused on the precise assignment questions. Do not go off on tangents or devote a lot of space to summarizing general background materials.
Use evidence from your readings to justify your conclusions.
Be sure to cite at least five credible resources.
Make sure to reference your sources of information with both a bibliography and in-text citations. See theStudent Guide to Writing a High-Quality Academic Paper, including pages 1114 on in-text citations. Another resource is the Writing Style Guide, which is found under My Resources in the TLC Portal.
Your assignment will be graded using the following criteria:
Assignment-Driven Criteria:Student demonstrates mastery covering all key elements of the assignment.
Critical Thinking/Application to Professional Practice:Student demonstrates mastery conceptualizing the problem and analyzing information. Conclusions are logically presented and applied to professional practice in an exceptional manner.
Business Writing and Quality of References:Student demonstrates mastery and proficiency in written communication and use of appropriate and relevant literature at the doctoral level.
Citing Sources:Student demonstrates mastery applying APA formatting standards to both in-text citations and the reference list.
Professionalism and Timeliness:Assignments are submitted on time.
Module 2 – SLP
DESIGNING THE STUDY AND ASSESSMENT OF THE RESEARCH CONTEXT
Now that you have described the context for your research and provided a description of the secondary data available for that firm, you will now begin to mine that data. As an example, the United Way of Orange County was selected to demonstrate some use of Excel for your background analysis. For nonprofits, there is a database listed in the secondary sources list that has the financial report forms 990 listed for the nonprofits. This data is used in a running analysis for the United Way of Orange County.
If you need an Excel refresher, here are some resources to help you:
Brown, N., Lave, B., Romey, J., Schatz, M., & Shingledecker, D. (2018)Beginning Excel. OpenOregon, Creative Commons License. Retrieved fromhttps://openoregon.pressbooks.pub/beginningexcel/.
Harvey, G. (2016).Excel 2016 all-in-one for dummies[Books24x7 version]. John Wiley & Sons.Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter112925in the search bar.
Harvey, G. (2016). Chapter 10: Charming charts and gorgeous graphics. InExcel 2016 for dummies[Books24x7 version]. John Wiley & Sons. Available in the Trident Online Library: Follow these instructions forFinding Skillsoft Books. Enter117498in the search bar.
In Figure 3 below is the Excel Chart done for the United Way of Orange County showing how the spreadsheet was set up. (Click on the image below to open the file.)
Figure 3.
Using this as a model and using the secondary data you found on your firm, produce a background analysis on the firm that shows some pertinent information in the spreadsheet along with charts to show the impact this firm is having in the area or the competitive space in which it is located. This is your unique perspective on the firm you have selected for the dissertation, and you will put this in the appendix ultimately as background on your firm. This, along with the Introduction you developed in SLP 1, will be the second part of the Background Analysis for your firm that will go in the Appendix of the dissertation. Integrate the spreadsheet and graphical analysis into your Module 2 SLP.
SLP Assignment Expectations
Your assignment will be graded using the following criteria:
Assignment-Driven Criteria:Student demonstrates mastery covering all key elements of the assignment.
Critical Thinking/Application to Professional Practice:Student demonstrates mastery conceptualizing the problem and analyzing information. Conclusions are logically presented and applied to professional practice in an exceptional manner.
Business Writing and Quality of References:Student demonstrates mastery and proficiency in written communication and use of appropriate and relevant literature at the doctoral level.
Citing Sources:Student demonstrates mastery applying APA formatting standards to both in text citations and the reference list.
Professionalism and Timeliness:Assignments are submitted on time.
Discussion: Modeling for Your Dissertation (300 words)
Modeling was the focus of this module.
What are the key factors (constructs) driving your dissertation research?
What relationship do you see between these constructs?
