165
Psychology (Fach) / Research Methods (Lektion)
Anzeige
- Effect size - Two families of effect size: > Correlational family: (phi, rpb, R², partial-eta squared…) > Difference family: (Cohen’s d, Hedge’s g…) - Interval estimates around effect sizes > null-counternull interval: based on the actual p value rather than on previously specified alpha > the ''null” anchoring one end of the interval is the effect size that is associated with the null hypothesis (and is typically zero) > counternull refers to the non-null magnitude of the effect size that is larger than the obtained effect size and is supported by the same amount of evidence that is supported by the null value of the effect size > this interval (null value to counternull value) can alert a researcher to whether a conclusion of “no effect” might be an error (type 1 error). -> effect size: zero = results should be non-significant, if they are significant anyways type 1 error-> effect size: large = results should be significant, if they aren't anyways type 2 error
- The scientific process - develop a research question - generate a research hypothesis - form operational defintions - choose a research design - evaluate the ethical issues - get funding - collect and analyze data, form conclusions - report research results
- Research ethics - key values - honesty and transparency - competence - scepticism - openness - fairness - collegiality
- 1947 Nuremburg Code - the nuremburg tribunal outlines 10 ethical standards for research with human participants: > consent > consideration of benefits and risks > participantion is voluntary > high quality of research designs - The Nuremberg Code is a set of research ethics principles for human experimentation set as a results of the Subsequent Nuremberg Trials at the end of the Second World War
- 1964 Declaration of Helsinki Developed by the World Medical Assosciation Last amended in 2008, provides the most widely accepted guidlines for research with humans - The participants well-being takes precedence over the interests of society and science - Written consent - Limited use of placebo - Greater acces to benefits
- APA Code of Conduct - Principle II: Beneficience and Nonmaleficience: doing good and avoiding harm > “Debriefing” if deception is used; also referred to as “dehoaxing”. Can be useful for disclosure of information that was not revealed before as well. - Principle V: Fidelity and Responsability/ Integrity > “Causism” is a problem that occurs if someone implies a causal relationship where none is supported by the underlying data. > a) ''Intentional plagiarism'' – deliberate copying or taking of someone else’s ideas or work and then knowingly failing to give credit or failing to place the quoted passage in quotation marks with a specific citation b) ''Accidental plagiarism'' – the use of someone else’s work but then innocently forgetting to credit it - Principle IV:Trust > “Confidentiality” is intended to ensure the subject’s privacy by procedures for protecting the data. - Principle III: Justice >“Placebo”/ ''Waiting List Control Group'' > “Consequentialist” view refers to the argument stating that right or wrong depends on the last consequence > ''Deontolgical'' view: some actions may be presumed to be categorically wrong no matter what the consequences/ what is to be considered right or wrong depends on the procedure more than the consequences - Principle I: Respect for Peoples Rights and Dignity > “Informed Consent” refers to the procedure in which prospective subjects voluntarily agree May also impair validity of the research (subject expectations) or can create doubts on its own (paranoid ideation). Principles that appear in all European and American Codes of Conduct 1. To avoid physical harm 2. To avoid psychological harm 3. To keep the data confidential
- APA Ethics Code - applies to research, teaching, therapy and administration
- Minimal Risk - harm or discomfort is not greater than what that experienced in daily life or during routine physical or psychological tests - minimal risk differs across participants - vs. ''at risk''
- Confidentiality - protection against social risk - confidentiality≠anonymity - how to: > report results in statistical averages > remove identifying information
- ''Risks'' - physical injury - psychological injury - social injury (embarrassment)
- Anzeige
- Informed consent - a social contract - Make clear to participants: > nature of the research (what they will do) > possible risks - Written informed consent > required when risk is greater than minimal > not required when researchers observe public behavior - provide assent if participant is not able to give consent > guardian's consent - able to withdraw at any time/ no pressure
- Deception Occurs when: - information is withheld from participants - participants are intentionally misinformed about aspects of the research Should be used when: - study is important and not studyable any other way - deception would not directly influence participation + > allows study of natural behaviour /unbiased by the study's cause - > contradicts principle of informed consent > relationship between subject and researcher is not open and honest > frequent deception makes people suspicious of research and psychology > ALWAYS unethical to use deception is recruiting participants
- Debriefing > especially necessary when deception is used > goal: participants should leave feeling good about the research experience + for participants – Explain the need for deception – Reverse any harmful effects – Education – Positive feelings + for researchers – Show whether the study was perceived as intended – Leads for future and current protocol issues – Identify errors
- Scientific Misconduct - defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results - Fabrication: making up data ad reporting it as real - Falsification: the manipulation of research materials, equipment, or processes, or the change or omission of data, such that the research is not accurately represented in the research record - Plagiarism: the appropriation of another person’s ideas, processes, results, or words without giving appropriate credit > micondust must be intentional, knowingly, recklessly - Identify misconduct > failure to replicate > peer review > whistleblowing: person reports alleged misconduct
- Publication Ethics - Authorship - substantial scholarly contribution: > substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data > drafting article of revising critically for intellectual content > final approval of the version to be published - Acquisition of funding, collection of data, or general supervision of the research group alone does not constitute authorship
- Publication Ethics - Author order - inconsistent across fields, but mostly consistent within fields > Lead author: overall responsability for the manuscript, main contributor to conception and write up of idea > Last author: typically head of the lab, contributed revisions and refinements of the manupscript and idea > Middle authors: helped with acquisition and analysis of data; may have made contribution to development of design and idea - roles should be made clear from the beginning - assign responsability and ensure accountability
- Wittgenstein vs Popper - Puzzles and Problems A controversy between Wittgenstein and Popper around their views of philosophy: - deconstructed philosophy to atomistic level - ''no problems'' in philosophy, merely ''linguistic puzzles'' revealing misuse of language (Wittgenstein)
- Ethics - the values by which the conduct of individuals is morally evaluated
- Institutional Review Board (IRB) - independent group of people overseeing the work of researchers
- Active Deception (comission) - the deception through intentional misleading information on behalf of a certain party - misrepresenting the purpose of the study or making false promises - less common that passive deception
- Passive Deception (omission) - the use of deception via leaving out or avoiding something - observing people without telling them; not offering disclosure
- The Belmont Report - Belmont Report: Includes emphasize on “respect”, maximization of plausible benefits (minimizing harm), and fairness. - If more than minimal risk is involved then specific safety requirements are necessary. - Minimal risk research: Including studies in which the probability for the participant to get harmed is not higher than in everyday life. - The emphasis is on 5 broad principles. > see APA Code of Conduct
- Reliability - the consistency or repeatability of the measurements
- Validity - the degree of accuracy to which a study reflects or assesses the specific concept that the researcher is attempting to measure - study measures what its supposed to measure
- Test- retest reliability > Reliability assess Stability and Equivalence: > from one measurement to another > the procedure for estimating the test-retest coefficient involves administering the instrument to a group of subjects and then administering it to the same group later > this type of reliability is important to measure trait characteristics (stable attributes) > Peason's r > > “Stability coefficients” are correlations between scores on the same form administered to the same people at different times > “Cross lagged correlations” are used to indicate both, the stability and the equivalence (same people, different times)
- Alternate Form Reliability > Equivalence - assess Stability and Equivalence: > refers to the degree of equivalence of different versions of an instrument with different components (e.g., test items) that are all intended to measure the same thing > the correlation between two tests is called “equivalent-forms reliability”/ forms are required to have the same variance > “Coefficients of equivalence'' are correlations between scores on different forms administered at the same time > “Cross lagged correlations” are used to indicate both, the stability and the equivalence (same people, different times)
- Internal-consistency reliability - tells us the degree of relatedness of the individual items or components of the instrument in question when we want to use those items to obtain a single score - is the degree of relatedness of items that measure the same thing in a test > “reliability of components” - internal-consistency reliability of a test too low -> increase n of items = increases R if items remain reasonably homogenous > the larger the number of items, the more likely it is that fatigue and boredom will result in inattention and careless responding - The Spearman-Brown equation > Kuder and Richardson’s formula 20 > Cronbach’s alpha coefficient
- Internal-consistency Reliability: Spearman Brown > can be rearranged to find out how many items the test would need to satisfy a certain amount of reliability > assumes that the items in the composite test, whose score is the simple total of all items, measure the same trait and have the same variance, and that all pairs have the same correlation = the more assumptions are violated the less the implications are of use
- Internal-consistency Reliability: Kudar-Richardson/Conbach's alpha Split-half reliability: obtained via correlating one half of a test with the other half. > the problem is that we get different values depending on where we do the split > KR20: based on all possible splits, and is used when test items are scored dichotomously > KR20: special case of Cronbach's alpha > Cronbach's alpha: more convinient for larger sample of questions/ not just for dichotomous scores
- Replication in research How > difference: precise (supports theory/damages theory) & varied (extends theory/limits theory) replication When > early replications are more valuable than later ones Who > independence of replicators
- Anzeige
- Interrater Agreement & Reliability Interrater agreement ≠ Interrater Reliability - Interrater agreement: .percentage of agreement of the judges .does not differentiate between accuracy and variability (agee because of same bias?) .better to use a measure of the product moment correlation > phi coefficient is a better measure (measures r of a 2x2 table) - Interrater reliability: .cohens kappa > adjusted for agreement based on lack of variability > problems with df > 1
- Cohen's Kappa > omnibus statistical procedures: df > 1> focused statistical procedures: df = 1 > kappa > the problem with most omnibus kappas is that we cannot distinguish which focused or specific judgments are made reliably and which are made unreliably > K and r are equivalent only when the row totals for levels A and B are identical to the column totals for levels A and B
- Cohen's Kappa > omnibus statistical procedures: df > 1> focused statistical procedures: df = 1 > kappa: takes into account the agreement occurring by chance > the problem with most omnibus kappas is that we cannot distinguish which focused or specific judgments are made reliably and which are made unreliably > K and r are equivalent only when the row totals for levels A and B are identical to the column totals for levels A and B
- Content-related Validity - requires that the items represent the material/content they should - the subjective evaluations of expert judges are usually required to assess this factor
- Criterion-related Validity - refers to the degree to which the test or questionnaire correlates with one or more relevant criteria - concurrent validity: when the measure is compared to a preexisting measure for the same criterion - shorter form of a test is compared with a longer version- longer version is the criterion in this cas - predictive validity: results of a test are saved and compared to a newer/future measure of the same criterion, when this measure is available ex. results of a high-school maths test and future college grades
- Construct Validity - a measure of the psychological characteristic and its correlation with the questionnaire that was used to assess it - Convergent validity: is achieved when the test correlates highly with different methods or measures that are thought to measure the same trait or behavior - Discriminant validity: is achieved when the test does not correlate with different methods or measures thought to measure an unrelated trait or unrelated behavior - The “Multitrait-multimethod matrix” of intercorrelations is used to triangulate the convergent and discriminant validity of a construct > the idea behind using multiple methods to measure the same and different traits is that it avoid the problem that high or low correlations may be due not to convergent or discriminant validity, but to their common biases in the same method of measurement
- Selection ratio - the proportion of applicants to be selected by a test - selection accuracy increases as the validity coefficients increase and the selection ratios decrease - benefits of increasing validity coefficients are usually greater as selection ratios decrease business example: >Calculation selection ration: number of applicants business plans to hire _________________________________ total number of applicants for positions >low ratio (close to 0) = many applicants for the job; judicous hiring process is possible, but highly qualified applicants have to be rejected
- Relationship Reliability/ Validity > high reliability & low validity: measure that is consistent in getting bad information or consistent in missing the mark > high validity & low reliability: not possible in practice you cannot have validitiy without reliability- An instrument that is valid is always reliable! > you cant really get at what you want or what youre interest in if your measure fluctuates wildly > low reliability & low validity: inconsistent measure and not on target
- Qualitative Data - refers to data in written from: - data that consistes of people's spoken words, recorded conversations, narrative responses in nondirective or unstructured interviews and observable behaviour - data that does not exist in some numerical form
- Quantative Data - refers to data in some numerical form/ numerical data
- Ethnographers - interested in cultures, and their preferred strategy of inverstigation is participant observation: interacting while participants as observers in a culture term is used liberally to embrace diverse strategies, flexible research designs and different levels of involvement - informants: indigenours memebers of the culture who provide the researcher with insider information - autoethnography: researchers sometimes study their own social scientific culture -
- Ethnographers- Data sampling methods > general distinction: observation of events vs observations of traits - Time sampling: sampling specified periods and recording everything of interest during that perios - Behavioural sampling: used when a particular ongoing behaviour is of interest; the researcher samples the behaviour periodically, typically using tally sheets and filed notes to keep records of a multitude of occurences
- Ethnographers - Field notes > written notes: verbatim quote or a researcher's paraphrase? > observations can only be made with the permission of the informant 1) condensed account: written notes consisting of single words, phrases and unconnected sentences quoted on the spot > signed by participants, in order to be used legally for the studies purposes 2) expanded account: fills in details from the recall of things that were not recordable earlier (fills in condensed account) 3) fieldwork journal: ethnographers written diary of particular experences, ideas, fears, mistakes and confusions (personal reactions to the events)
- Noninteractional artifacts - systematic errors that operate in the eye of the scientist but are not due to uncontrolled variables that might interact with the subjects behaviour > interpreter bias and observer bias
- Interpreter Bias - refers to systematic errors that occurs during the interpretation-of-data phase of the research process > i.e: clining to a specific theory > control for interpreter bias: make the results available to other researchers, who are free to agree/disagree with the interpretation
- Observer Bias - refers to systematic errors in the observation or recording phase of the research > Problem: what we see is not necessarily what actually happens, but just what we percieve something to be like taking our experiences and assumptions into account > perception ≠ reality
- Reactive measurements - measurements/observational researchs that affects the behaviour that is being measured
- Nonreactive measures - measurements/observational research that does not affect the behaviour being measured > Archives > Content analysis > Physical traces > Simple unobstrusive observation
- Nonreactive measures - Archives two categories: > running records - actuarial data (birth, marriage etc.), political and judical information, mass media (facebook timeline) > official dokuments/public documents > personal documents and episodic records - diaries and letters, rumors that people phoned in the rumor control center, photgraph and picture postcards, vidual presentations in the picture books
- Nonreactive measurements - Content Analysis > the name given to a set of procedures that are used to categorize and evaluate pictorial, verbal or textual material > use ''something'' (hypothesis, logic) as a basis to ''something'' 1) sorting of data must be consistent among judges 2) specific categories and units chosen > must be relevant to the question of interest 3) if not all material available is to be analyzed: use appropriate sampling procedure