Planning your research: Reviews, hypotheses, and ethical pitfalls

Август 4, 2022

Главная
Педагогика
Planning your research: Reviews, hypotheses, and ethical pitfalls

Содержание

2. Today’s Questions What decisions do we make as we plan our research? How to do a
3. What does a research begin with? Research problem, or a research question. Any question (which may
4. Research stages Research question! Methods: - what?.. - how?.. - where?.. - in whom?.. …shall we
5. Phenomenon What research questions can you think of?
6. Research problem Is a research problem a scientific problem? Depends on: Is it formulated using scientific
8. Doing a Theoretical Review: How to make it a (relatively) painless process
9. Aim of the study. A study can be… Exploratory (looking for associations, describe phenomena to formulate
10. The Place of Theory in Research Two positions concerning the place of theory: Theory ? Problem
11. Three levels of theory (Madsen, 1988)
12. Madsen, 1988 Hypothetical constructs, trans-empirical terms, research questions Measurable variables (latent and directly observed), empirical hypotheses
13. Trans-empirical terms Personality – … Common sense: a human being; General scientific sense: the combination of
14. The danger of everyday language The same common language term can denote very different psychological processes
15. Doing Literature Reviews
16. Why theoretical reviews? Make sure what you want to do is up to date = you
17. Theoretical Reviews Theoretical review as a basis for an empirical study has to justify the study
18. Original Substantive Contribution = message: Replication: “The field is in the right place” Redefinition (of the
19. Sternberg: Quality criteria for theories Clarity and Detail: is it clear what it says? Relation to
20. A good review has Wide scope Depth of analysis Relevant sources Careful interpretations Includes critical analysis
21. Structuring your review Theoretical logic: general points of a theory ? specific theories / models ?
22. Review flaws Ignoring sources (happens often) Misinterpretation (is more likely to happen when you rely on
23. Don’t be afraid of re-writing!
24. Plagiarism Plagiarism is using in your own work other people’s results, formulations or ideas without referencing
25. How to avoid plagiarism? Make sure that ideas and facts you refer to, except for common
26. «Antiplagiat» (Turnitin, …) «Percentage of original text» says very little about the quality of a work,
27. Steps in doing a lit review Define problem not too wide, not too narrow Set your
29. How to get a quick overview of a topic? Library.hse.ru – Electronic resources ? Scopus Enter
30. Lit Search Algorithm Find papers in Scopus / ISI Web of Science. Use HSE_FullText button to
31. Structuring your review Sort papers in folders Create files with abstracts Use reference managers: Mendeley (http://www.mendeley.com)
32. Questions to assess lit. reviews Does the review give a comprehensive information about the way problem
33. Operationalizing = going from theory to hypotheses and methods
34. From a research question to a hypothesis A research problem can be rather abstract, not always
35. Definitions When we formulate our hypotheses, we need to give operational definitions for the concepts based
36. Operational definition The construct Operational definition (depends on research question)
37. Hypotheses Theoretical hypotheses (test logically by theoretical analysis) Empirical hypotheses (test empirically): Existence of a phenomenon;
38. Evaluating hypotheses Are they clear and unambiguous? Are they testable? Are they grounded in a theoretical
39. Methods choices What and where shall we study? (Operationalization choices) What phenomena? (consciousness, behavior, …) Using
40. The choice of a research question is related to the choice of an approach «Quantitative» questions
41. A Primer on Research Ethics before you start investigating
42. Ethical Considerations Why is research ethics important? Ethical standards in psychology exist for: Researchers Publication authors
43. Aims of research ethics Protecting the physical and mental health of individuals (and animals) participating in
44. Care about participants Principles (Belmont protocol): Respect for person: Treat people as autonomous agents ? Provide
45. Research Ethics Committees IRB: Institutional Review Boards – do they help? IRB
46. Care about respondents The practical means used in psychology research: Providing choice ? Informed consent; Ensuring
48. Informed consent includes: Description of research (aims, requirements, procedure, compensation) Description of risks and benefits (if
50. Privacy and confidentiality in research We infringe privacy when: we collect information about individuals which, if
51. Privacy / confidentiality advice Whenever you can avoid collecting identifying information (name, etc.), it is better
52. Deception Deception is giving imprecise or misleading information about study aims before the study. Is justified
53. Ethical standards in test use (ITC) General (in any context) Professionalism (do not use tools you
54. Unethical Behavior in science Violations against authorship / copyright: Plagiarism; Collusion (wrong authorship credit, ghostwriting); Using
55. APA publication guidelines
56. Ethics checklist Did you use procedures to protect the rights of participants? autonomy ? informed consent;
58. Скачать презентацию

Слайд 2

Today’s Questions
What decisions do we make as we plan our research?
How

to do a good literature review?
Before you start: how to avoid ethical pitfalls?

Слайд 3

What does a research begin with?
Research problem, or a research question.
Any question

(which may even seem weird), concerning some mental phenomenon or process.

Слайд 4

Research stages
Research question!
Methods: - what?.. - how?.. - where?.. - in whom?..
…shall we study?
Data collection

?? Data analysis

Publish and move on!..

(lit. review)

Operationalizing

Слайд 5

Phenomenon
What research questions can you think of?

Слайд 6

Research problem
Is a research problem a scientific problem?
Depends on:
Is it formulated

using scientific concepts, does it refer to a scientific view of reality? (are the reviewers going to treat it as a nonsense?)
Is it related to existing theories, does it seem relevant within current scientific discourse? (however, you have a little chance of starting a paradigm shift)
Is it important for society? (would anyone be willing to give you money to do this research?)

Слайд 7

Слайд 8

Doing a Theoretical Review:
How to make it a (relatively) painless process

Слайд 9

Aim of the study. A study can be…
Exploratory (looking for associations,

describe phenomena to formulate theory)
Confirmatory (based on a theory, test a specific hypothesis or reproduce findings)
Critical (an outcome of the study resolves a competition between two or more different theories)

Слайд 10

The Place of Theory in Research
Two positions concerning the place of

theory:
Theory ? Problem ? Choose Phenomena ? Empirical Study ? Interpret Results = traditional strategy
Phenomenon ? Problem ? Empirical Study ? Interpret Results ? Theory = phenomenological (exploratory) strategy

However, in any case you still need review to know:
What other people have done
How they did it
What conclusions they arrived at?

Слайд 11

Three levels of theory (Madsen, 1988)

Слайд 12

Madsen, 1988
Hypothetical constructs, trans-empirical terms, research questions
Measurable variables (latent and directly observed),

empirical hypotheses

-------- the gap of operationalization --------

Слайд 13

Trans-empirical terms
Personality – …
Common sense: a human being;
General scientific sense: the

combination of all individual differences;
Narrow sense: whatever a certain personality theory says it is: e.g., subject of needs, subject making decisions, etc.
R. B. Cattell: personality is like love: everyone knows that it is, but no one knows what it is.
It is not a data term, but something different: a ‘trans-empirical term’ (Madsen) or a ‘metapsychological category’ (Petrovsky & Yaroshevsky).

Слайд 14

The danger of everyday language
The same common language term can denote

very different psychological processes (“love”, “conscience”, “personality”…)
Even a clearly defined scientific construct can often be expressed in many very different everyday terms (“extraversion”)
We should not completely rely on self-report data but interpret it:
e.g. “– I love him – What do you mean by love/feel?”
Dmitry Leontiev: “The difference between sociologists and psychologists is that sociologists do believe in whatever people say, and psychologists do not”.

Слайд 15

Doing Literature Reviews

Слайд 16

Why theoretical reviews?
Make sure what you want to do is up

to date = you need to avoid inventing the bicycle.
Look at different ways to formulate your problem theoretically and to study it empirically = find out their strong and weak points.
Generalize the existing theoretical and accumulated empirical data = what is important today (or tomorrow)?

Слайд 17

Theoretical Reviews
Theoretical review as a basis for an empirical study has

to justify the study by answering questions like:
what it is that you are trying to study, how it can be defined?
why is it necessary to study this? has anyone done it before?
why do you choose this experimental paradigm?
Theoretical review as a special type of analytic work:
clarifies the way a problem is stated and studied in science;
combines and generalizes existing studies as a digest for readers;
reveals connections, contradictions, «blind spots» and inconsistencies in existing literature;
shows next steps to be made in the solution of a problem. (Eisenberg, 2000).

Слайд 18

Original Substantive Contribution = message:
Replication: “The field is in the right

place”
Redefinition (of the current status of the field)
Incrementation (a step forward)
Advance Forward (before others are ready)
Redirection (of the field)
Reconstruction & redirection (restart from past)
Reinitiation (start from a new point)
Integration (diverse ways of thinking ? unify)

Sternberg: Quality criteria for reviews & theories

Слайд 19

Sternberg: Quality criteria for theories
Clarity and Detail: is it clear what

it says?
Relation to Past Work: does it build on past?
Falsifiability: does it make empirical predictions?
Generalizability: in what situations does it work?
Discriminability: does it include its limitations?
Internal Consistency: is it logically coherent?
Correspondence to Past Data: fit or selective fit?
Prediction: does it fit future data?
Parsimony: is it simple enough?
Excitement: is it exciting or boring?

Слайд 20

A good review has
Wide scope
Depth of analysis
Relevant sources
Careful interpretations
Includes critical analysis
Makes

conclusions
Is logically structured (A->B->C)
Is effective: information/volume

Слайд 21

Structuring your review
Theoretical logic: general points of a theory ? specific

theories / models ? empirical findings…
Historical logic: Plato ? … ? Wundt ? … ? Your supervisor
The logic of phenomena: there is A, there is B ? their relationship ? a research problem
«As you like»: Nancy Eisenberg: there is no ‘right’ way to structure a literature review.

Слайд 22

Review flaws
Ignoring sources (happens often)
Misinterpretation (is more likely to happen when

you rely on secondary sources, like textbooks, existing reviews, etc.)
Selective quotation (unethical in science)
Misrepresentation of facts (completely unscientific) (Newby, 2010)

Слайд 23

Don’t be afraid of re-writing!

Слайд 24

Plagiarism
Plagiarism is using in your own work other people’s results, formulations

or ideas without referencing a source (? appropriation: they are impossible to tell from your original work).
Plagiarism can be unintentional (because of improper or absent referencing), as well intentional.
«Self-plagiarism»: double publication of one’s own results (without referencing) or re-using one’s existing texts in a supposedly new work (without citing or acknowled).
Plagiarism is a violation of academic integrity ? sanctions.
http://turnitin.com/assets/en_us/media/plagiarism-spectrum/#.V8ZO8OOTAqk.facebook

Слайд 25

How to avoid plagiarism?
Make sure that ideas and facts you refer

to, except for common knowledge [e.g., secondary school course], are provided with references to their sources.
Make sure you are allowed to re-use fragments of your old work or your old data; provide references.
Correct citations:
verbatim: «”Clearly, the Earth is round,” wrote Ivanov (1988, p. 23)»;
paraphrase: «Ivanov (1988) suggested that Earth is round».
reference without quoting: «The round-Earth position is shared by Ivanov (1988), Petrov (1989), and Sidorov (2012)».

Слайд 26

«Antiplagiat» (Turnitin, …)
«Percentage of original text» says very little about the

quality of a work, because it does not differentiate between legitimate citations and plagiarism.

Слайд 27

Steps in doing a lit review
Define problem
not too wide, not too

narrow
Set your questions
Choose a range of sources
Travel, following references
Make abstracts, if needed
Establish a structure
Analyze and generalize

Слайд 28

Слайд 29

How to get a quick overview of a topic?
Library.hse.ru – Electronic

resources ? Scopus
Enter keywords
Sort articles by citations
Look at first 10-20-… (depending on how much time you have) paper, paying more attention to reviews

Слайд 30

Lit Search Algorithm
Find papers in Scopus / ISI Web of Science.
Use

HSE_FullText button to arrive at papers.
If it does not work, use «A-to-Z сводный каталог» to find out whether our library subscribes a journal.
Use Google Scholar (wider scope: e.g., preprints, dissertations and other unpublished works, but more rubbish).
Use РИНЦ (elibrary.ru) Russian Index of Scientific Citations to look for Russian-language works.

Слайд 31

Structuring your review
Sort papers in folders
Create files with abstracts
Use reference managers:
Mendeley

(http://www.mendeley.com)
Zotero (http://www.zotero.org)
(they store papers and abstracts, creating reference lists automatically in different standards, e.g., ГОСТ or APA)

Слайд 32

Questions to assess lit. reviews
Does the review give a comprehensive information

about the way problem has been studies, does it take into account main approaches and methods to solve it?
Is the review a sufficient justification for a study: does it show that this study needs to be carried out, and in this way?
Is the review economical (concise), structured, and readable?

Слайд 33

Operationalizing
= going from theory to hypotheses and methods

Слайд 34

From a research question to a hypothesis
A research problem can be

rather abstract, not always testable
A hypothesis – is a general, but exact statement about reality:
formulated in scientific terms (not everyday terms), based in some understanding of reality;
the verisimilitude (probability of being true) of a hypothesis can be tested either by logical analysis (theoretical hypothesis) or by an empirical proceduce (empirical hypothesis).
A good hypothesis can be tested. A bad hypothesis can not be tested.
(A good hypothesis: it is also not clear whether it’s right or wrong…)

Слайд 35

Definitions
When we formulate our hypotheses, we need to give operational definitions

for the concepts based on some theories or some phenomena.
Operational definition of a construct refers to measurable variables (data stratum) and is always limited, compared to its theoretical definition:
E.g., how can we operationalize aggression? = What exactly would we measure/observe/record in a study?

Слайд 36

Operational definition
The construct
Operational definition
(depends on research question)

Слайд 37

Hypotheses
Theoretical hypotheses (test logically by theoretical analysis)
Empirical hypotheses (test empirically):
Existence of

a phenomenon;
Correlation between phenomena;
Causal association between phenomena.
Statistical hypotheses (in terms of measured variables):
Null hypothesis (H0): «No effect».
Alternative hypothesis (H1): «The null hypothesis is wrong».
In an exploratory study, a research question without explicit hypothesis may be sufficient.

Слайд 38

Evaluating hypotheses
Are they clear and unambiguous?
Are they testable?
Are they grounded in

a theoretical context (and why in this one)?
What other possibilities for operationalization of these hypotheses exist (and why this one is chosen)?

Слайд 39

Methods choices
What and where shall we study? (Operationalization choices)
What phenomena? (consciousness,

behavior, …)
Using what measurement procedures? (? data type)
In which setting?
Using what sample?
How shall we study it? (Design choices)
What is the study plan (experiment, etc.)?
What data analysis methods shall we use?
What exactly shall we do?
Procedure (protocol)

Слайд 40

The choice of a research question is related to the choice

of an approach

«Quantitative» questions

Is there a causal link between X and Y?
Do people with different X differ in Y? (association)

«Qualitative» questions

How…? (? describe the situation, experience)
Why…? (? describe the variety of goals, intentions)

Слайд 41

A Primer on Research Ethics before you start investigating

Слайд 42

Ethical Considerations
Why is research ethics important?
Ethical standards in psychology exist for:
Researchers
Publication

authors
Test developers / users
Practitioners (therapists, counsellors) [we will not look into these]

Слайд 43

Aims of research ethics
Protecting the physical and mental health of individuals

(and animals) participating in research.
Protecting privacy and/or ensuring confidentiality of information.
Ensuring the scientific data is correct (academic integrity).

Слайд 44

Care about participants
Principles (Belmont protocol):
Respect for person:
Treat people as autonomous agents

? Provide choice
Protect those with diminished autonomy
Beneficence:
Do not harm ? Maximize benefits for people, minimize risks
Justice (mainly applies to medical research):
Select people fairly.

Слайд 45

Research Ethics Committees
IRB: Institutional Review Boards – do they help?
IRB

Слайд 46

Care about respondents
The practical means used in psychology research:
Providing choice

? Informed consent;
Ensuring confidentiality ? Data protection;
Reducing the harmful consequences of deception ? Debriefing.

Слайд 47

Слайд 48

Informed consent includes:
Description of research (aims, requirements, procedure, compensation)
Description of risks

and benefits (if any), and of ways risks will be managed
Explicit notification that a person is free to withdraw from the study at any time without any negative consequences for him/her
Even if students are required to take part in studies, there needs to be a choice of available research projects
Contacts of researchers (for questions) and ethic committee (for complaints)

Слайд 49

Слайд 50

Privacy and confidentiality in research
We infringe privacy when:
we collect information

about individuals which, if disclosed, could harm their reputation, social status, employability, endanger them, etc.
and this information is collected together with data that make individuals identifiable.
If both “yes”, then we need to care about Confidentiality:
take measures to protect the information from disclosure

Слайд 51

Privacy / confidentiality advice
Whenever you can avoid collecting identifying information (name,

etc.), it is better to do so.
E-mails and IP addresses may also be considered identifying information
If you do collect such information, make sure you anonymize your data afterwards
Keep identifiers separately from data (and safely = in a restricted-access, protected way)

Слайд 52

Deception
Deception is giving imprecise or misleading information about study aims before

the study.
Is justified in case when it would be impossible to perform the study without using it.
Whenever deception is used, participants must be debriefed after the study:
unless debriefing results in more harm: e.g., you selected them based on some unpleasant property, like overweight, etc.

Слайд 53

Ethical standards in test use (ITC)
General (in any context)
Professionalism (do not

use tools you are not trained in)
Responsibility (only use tests for their proper aims)
Competence (make limited interpretations)
Fairness (use correct and group-specific test norms)
Security (of test materials) and confidentiality (of results)
Research-specific
Obtain permissions (for use or re-printing)
Document (describe) measures and any modifications made
Prevent research tools (in progress) from spreading into practice

Слайд 54

Unethical Behavior in science
Violations against authorship / copyright:
Plagiarism;
Collusion (wrong authorship credit,

ghostwriting);
Using products of other people’s work without permission.
Violations against scientific integrity:
Self-plagiarism;
Selective publication;
Data fabrication.

Слайд 55

APA publication guidelines

Слайд 56

Ethics checklist
Did you use procedures to protect the rights of participants?

autonomy ? informed consent;
information ? debriefing;
privacy ? confidentiality, data protection.
Have you ensured the academic integrity is not violated?
the data are correct and described in a complete manner;
conflicts of interest are disclosed.
Have you ensured copyright is not violated?
no plagiarism;
have permissions to use other people’s instruments, pictures, etc.
authorship and affiliations are stated correctly.
Do you need (have) an IRB (Ethics committee) approval?

Planning your research: Reviews, hypotheses, and ethical pitfalls

Содержание

Today’s QuestionsWhat decisions do we make as we plan our research?How

What does a research begin with?Research problem, or a research question.Any question

Research stagesResearch question!Methods: - what?.. - how?.. - where?.. - in whom?..…shall we study? Data collection

PhenomenonWhat research questions can you think of?

Research problemIs a research problem a scientific problem?Depends on:Is it formulated

Doing a Theoretical Review:How to make it a (relatively) painless process

Aim of the study. A study can be…Exploratory (looking for associations,

The Place of Theory in ResearchTwo positions concerning the place of

Three levels of theory (Madsen, 1988)

Madsen, 1988Hypothetical constructs, trans-empirical terms, research questionsMeasurable variables (latent and directly observed),

Trans-empirical termsPersonality – …Common sense: a human being;General scientific sense: the

The danger of everyday languageThe same common language term can denote

Doing Literature Reviews

Why theoretical reviews?Make sure what you want to do is up

Theoretical ReviewsTheoretical review as a basis for an empirical study has

Original Substantive Contribution = message:Replication: “The field is in the right

Sternberg: Quality criteria for theoriesClarity and Detail: is it clear what

A good review hasWide scopeDepth of analysisRelevant sourcesCareful interpretationsIncludes critical analysisMakes

Structuring your reviewTheoretical logic: general points of a theory ? specific

Review flawsIgnoring sources (happens often)Misinterpretation (is more likely to happen when

Don’t be afraid of re-writing!

PlagiarismPlagiarism is using in your own work other people’s results, formulations

How to avoid plagiarism?Make sure that ideas and facts you refer

«Antiplagiat» (Turnitin, …)«Percentage of original text» says very little about the

Steps in doing a lit reviewDefine problemnot too wide, not too

How to get a quick overview of a topic?Library.hse.ru – Electronic

Lit Search AlgorithmFind papers in Scopus / ISI Web of Science.Use

Structuring your reviewSort papers in foldersCreate files with abstractsUse reference managers:Mendeley

Questions to assess lit. reviewsDoes the review give a comprehensive information

Operationalizing= going from theory to hypotheses and methods

From a research question to a hypothesisA research problem can be

DefinitionsWhen we formulate our hypotheses, we need to give operational definitions

Operational definitionThe constructOperational definition(depends on research question)

HypothesesTheoretical hypotheses (test logically by theoretical analysis)Empirical hypotheses (test empirically):Existence of

Evaluating hypothesesAre they clear and unambiguous?Are they testable?Are they grounded in

Methods choicesWhat and where shall we study? (Operationalization choices)What phenomena? (consciousness,