How to Read a Study

Why This Matters

When evaluating peptide claims, the ability to critically assess scientific studies is invaluable. Marketing materials, social media posts, and online forums often cite studies selectively, misrepresent findings, or conflate animal research with proven human efficacy. This guide provides a practical framework for evaluating research on your own, so you can distinguish genuine evidence from hype.

You do not need a science degree to evaluate studies — you need a systematic approach and an understanding of the key concepts. By the end of this guide, you should be able to pick up a scientific paper, identify its strengths and weaknesses, and determine how much weight to give its conclusions.

The Structure of a Scientific Paper

Most research papers follow a standardized format known as IMRAD (Introduction, Methods, Results, and Discussion). Understanding this structure helps you know where to look for specific information.

Title and Authors

The title should clearly describe what was studied and how. Look at the author affiliations — are they from reputable institutions? Is the research group known for working on this topic? For peptide research, note if all authors are from the same institution (which may indicate a single-lab finding not yet replicated elsewhere).

Abstract

A brief summary (usually 200-300 words) of the study's purpose, methods, results, and conclusions. The abstract is useful for a quick overview but it frequently omits important nuances, limitations, and negative findings. Never evaluate a study based solely on its abstract.

Structured vs. unstructured abstracts: Many journals require structured abstracts with labeled sections (Background, Methods, Results, Conclusions). These are generally more informative and easier to parse than unstructured narrative abstracts.

Introduction

Provides background context, states the research question or hypothesis, and explains why the study was conducted. This section should clearly identify a gap in knowledge that the study aims to address.

What to look for: Does the introduction accurately and fairly represent the existing literature, or does it selectively cite studies that support the authors' hypothesis while ignoring contradictory evidence?

Methods

The most important section for assessing study quality. This section describes exactly how the study was conducted and should contain enough detail for another researcher to replicate the experiment.

Critical elements to check:

Study design (RCT, cohort, case series, animal study, in vitro)
Population (who was included and excluded, and why)
Intervention details (dose, route, frequency, duration)
Control group (placebo, active comparator, or none)
Randomization method and allocation concealment
Blinding (who was blinded — participants, clinicians, outcome assessors)
Primary and secondary outcomes (predefined or post-hoc)
Sample size justification (power calculation)
Statistical analysis plan
Ethical approval and informed consent

Results

Presents the data, ideally with tables, figures, and statistical analyses. This section should present all prespecified outcomes, not just the significant ones.

What to look for: Are the results consistent with the methods section? Are all primary endpoints reported? Are confidence intervals provided alongside p-values? Are adverse events reported?

Discussion

The authors' interpretation of results, placed in the context of existing literature. This is the most subjective section and should be read critically.

What to look for: Do the conclusions follow logically from the data? Do the authors acknowledge limitations? Do they overstate implications? Do they appropriately discuss the generalizability of their findings?

Conflict of Interest and Funding

Usually at the end of the paper. Look for disclosures of industry funding, consulting fees, stock ownership, or other relationships that could influence the results or interpretation.

Understanding Study Design

Randomized Controlled Trials (RCTs)

The gold standard for evaluating therapeutic interventions. Key subtypes:

Parallel group: Participants are randomly assigned to one of two or more treatment groups and remain in that group for the duration. The most common design.

Crossover: Each participant receives both treatments in sequence (separated by a washout period), serving as their own control. Increases statistical power with fewer participants, but only suitable when the condition is stable and the treatment effect is reversible.

Factorial: Tests two or more treatments simultaneously. For example, a 2x2 factorial trial might randomize patients to: (A) peptide + exercise, (B) peptide + no exercise, (C) placebo + exercise, (D) placebo + no exercise. Efficient for evaluating interactions between treatments.

Non-inferiority: Designed to show that a new treatment is "not worse" than an existing treatment by more than a predefined margin, rather than showing superiority. Common when the new treatment offers other advantages (convenience, cost, fewer side effects).

Cluster randomized: Groups (clinics, hospitals, communities) rather than individuals are randomized. Used when individual randomization is impractical.

Blinding

Open-label: Everyone knows who gets what. Most susceptible to bias, especially for subjective outcomes.

Single-blind: Participants do not know their assignment, but investigators do. Reduces participant-expectation effects but investigators may still influence outcomes.

Double-blind: Neither participants nor investigators know assignments. The standard for minimizing bias. Unblinding occurs only after data collection is complete.

Triple-blind: Participants, investigators, and data analysts are all blinded. The most rigorous approach.

Why blinding matters for peptides: Many peptide claims involve subjective outcomes (pain reduction, cognitive enhancement, sleep quality, energy levels, sense of well-being). These are highly susceptible to placebo effects. Without proper blinding, it is nearly impossible to separate a real drug effect from expectation effects. Injection itself has a strong placebo effect — simply receiving an injection (even of saline) can produce measurable improvements in pain and subjective well-being.

Observational Study Designs

Prospective cohort study: Researchers identify a group of people, measure their exposures (e.g., peptide use), and follow them forward in time to see who develops the outcome of interest. Stronger than retrospective designs because data is collected as events happen.

Retrospective cohort study: Uses existing records (medical charts, databases) to look back at exposures and outcomes. Faster and cheaper but limited by the quality of existing data.

Case-control study: Identifies people with an outcome (cases) and without (controls), then looks backward to compare exposures. Useful for rare diseases but susceptible to recall bias.

Cross-sectional study: Measures exposure and outcome at a single point in time. Can show associations but cannot determine temporal sequence (did the exposure come before the outcome?).

Sample Size and Statistical Power

Why Sample Size Matters

Larger studies are generally more reliable. Small studies are more susceptible to random variation and are more likely to produce false positives (detecting effects that do not really exist) or false negatives (failing to detect effects that do exist).

Power Analysis

Before a study begins, researchers should calculate the sample size needed to detect a clinically meaningful effect with adequate probability. This is called a power analysis and depends on:

Expected effect size: How large the treatment effect is anticipated to be (based on prior studies or pilot data)
Significance level (alpha): Usually set at 0.05
Power (1 - beta): The probability of detecting a true effect, conventionally set at 0.80 (80%) or 0.90 (90%)
Variability: How much the outcome measure varies between individuals

A study that is "underpowered" (too small) may miss a real effect and conclude the treatment does not work, when in fact the study simply lacked sufficient participants to detect it. Conversely, an extremely large study can find statistically significant differences that are too small to be clinically meaningful.

Red flag: If a study does not mention a power calculation or sample size justification, this is a methodological concern, particularly for studies that report negative results.

Primary vs. Secondary Endpoints

Primary Endpoint

The main outcome measure that the study was designed and powered to detect. This should be predefined in the study protocol and ideally registered on ClinicalTrials.gov before the study begins. The primary endpoint drives the sample size calculation and is the basis for the study's main conclusion.

Secondary Endpoints

Additional outcome measures of interest. These are typically exploratory and should be interpreted with more caution. A study that fails on its primary endpoint but succeeds on a secondary endpoint has fundamentally failed — the secondary finding should be considered hypothesis-generating, requiring confirmation in a future trial designed to test that specific outcome.

Post-Hoc Analyses

Analyses not planned before the study began, conducted after looking at the data. These are the least reliable because researchers can (consciously or unconsciously) test many outcomes and report only those that appear significant. Post-hoc findings are strictly hypothesis-generating.

Red flag in peptide research: If a study tested a peptide for one primary outcome, found no significant effect, but reports a significant finding on a secondary or post-hoc outcome, be cautious. This is often how marginal results are made to appear positive.

Intention-to-Treat vs. Per-Protocol Analysis

Intention-to-Treat (ITT)

All randomized participants are included in the analysis according to their original group assignment, regardless of whether they completed the study, adhered to the protocol, or even received the treatment. ITT preserves the benefits of randomization and provides a real-world estimate of treatment effectiveness.

Per-Protocol (PP)

Only participants who completed the study according to the protocol are included. This estimates the treatment's efficacy under ideal conditions but can introduce bias if dropouts are not random (e.g., if patients who experience side effects drop out of the treatment group, the remaining participants are a selected, potentially more tolerant subset).

Modified Intention-to-Treat (mITT)

A common compromise that excludes participants who never received any treatment or who had no post-baseline measurements. The exact definition varies between studies, which can complicate comparisons.

Best practice: Both ITT and PP analyses should be reported. If they agree, confidence in the results increases. If they disagree substantially, the reasons should be explored.

Understanding P-Values

What a P-Value Is

The p-value is the probability of observing results at least as extreme as those obtained, assuming the null hypothesis (no treatment effect) is true.

P = 0.05 means: "If the treatment truly has no effect, there is a 5% chance of seeing results this extreme or more extreme by random chance alone."
P = 0.001 means the probability is 0.1%.

What a P-Value Is NOT

Not the probability that the hypothesis is true or false. A p-value of 0.03 does not mean there is a 97% probability the treatment works.
Not a measure of effect size. A highly significant p-value (e.g., 0.0001) does not mean a large effect. With a very large sample size, even trivial effects become statistically significant.
Not a measure of clinical importance. Statistical significance and clinical significance are different concepts.
Not a measure of replicability. A p-value of 0.04 does not mean there is a 96% chance the finding will replicate.

The Multiple Comparisons Problem

If a study tests 20 independent outcomes at the 0.05 significance level, approximately 1 will be "significant" by chance alone — even if the treatment has no real effect. This is known as the multiple comparisons problem.

Correction methods: Bonferroni correction (divide alpha by the number of tests), Holm-Bonferroni (sequential adjustment), Benjamini-Hochberg (controls false discovery rate). If a study tests many outcomes without mentioning correction for multiple comparisons, this is a red flag.

P-Hacking

The practice of manipulating data analysis until a significant result appears. Techniques include: testing many outcomes and reporting only significant ones, adding or removing participants, adding covariates until significance is achieved, transforming data, and changing the endpoint after seeing preliminary results. P-hacking can be intentional or unconscious.

Confidence Intervals

A 95% confidence interval (CI) provides a range within which the true effect likely falls. It conveys both the magnitude and precision of the estimate.

Example: A study reports that a peptide reduces healing time by 3.2 days (95% CI: 1.5 to 4.9 days, p = 0.002).

This tells us:

The best estimate of the effect is 3.2 days faster healing
We can be 95% confident the true effect is between 1.5 and 4.9 days
The result is statistically significant (the CI does not cross zero)

Contrast: Another study reports a 3.2-day improvement (95% CI: -0.5 to 6.9 days, p = 0.09). Same point estimate, but the wide CI crossing zero tells us the result is imprecise and not significant — the true effect could plausibly be zero or even negative.

Why CIs are more informative than p-values alone: CIs show the range of plausible effect sizes, helping you judge clinical relevance. A "significant" result with a CI of 0.1 to 0.3 days improvement is statistically real but clinically trivial.

Absolute vs. Relative Risk Reduction

Relative Risk Reduction (RRR)

The proportional decrease in risk. If the control group has a 10% event rate and the treatment group has a 5% event rate, the RRR is 50%.

Absolute Risk Reduction (ARR)

The simple difference in event rates. In the example above, the ARR is 10% - 5% = 5 percentage points.

Why This Distinction Matters

Relative measures can be dramatically misleading. If the control group has a 0.2% event rate and the treatment group has a 0.1% event rate, the RRR is still 50% (sounds impressive) but the ARR is only 0.1% (one in a thousand patients benefits). Marketing materials almost always use relative risk reductions because they sound more impressive.

Always look for absolute numbers. If a study only reports relative risk reductions, calculate the absolute reduction yourself from the event rates.

Number Needed to Treat (NNT) and Number Needed to Harm (NNH)

NNT

The number of patients who must be treated for one additional patient to benefit compared to control. Calculated as 1 / ARR.

NNT = 1: Every patient benefits (essentially impossible)
NNT = 5: Treat 5 patients; 1 benefits beyond what placebo would provide
NNT = 50: Treat 50 patients for 1 to benefit
NNT = 100+: Marginal clinical benefit

Context matters: An NNT of 20 for preventing death is very different from an NNT of 20 for reducing mild headache frequency. The severity of the outcome being prevented must be weighed.

NNH

The number of patients treated before one experiences a specific adverse event. Calculated similarly to NNT but using harm rates. The ideal treatment has a low NNT and a high NNH.

Understanding Forest Plots

Forest plots are the standard graphical display in meta-analyses. They show the results of individual studies and the combined (pooled) estimate.

How to read a forest plot:

Each horizontal line represents one study. The square in the middle is the point estimate (the study's result). The size of the square reflects the study's weight (larger studies get larger squares). The horizontal line through the square is the 95% CI.
The vertical line at 0 (for differences) or 1.0 (for ratios) represents "no effect."
The diamond at the bottom represents the pooled estimate from all studies. Its width is the 95% CI.
If a study's CI crosses the no-effect line, that individual study is not statistically significant.
If the diamond does not cross the no-effect line, the pooled result is statistically significant.

Heterogeneity: The I-squared statistic measures how much the results vary between studies beyond what would be expected from chance. I-squared greater than 50% indicates substantial heterogeneity, meaning the studies may not be measuring the same thing, and pooling them may be inappropriate.

Funnel Plots and Publication Bias

A funnel plot graphs each study's effect size against its precision (usually standard error or sample size). In the absence of bias, points should form a symmetric funnel shape: larger, more precise studies cluster near the average, while smaller studies scatter more widely but symmetrically.

Asymmetry in funnel plots suggests publication bias — specifically, that small studies with negative results are missing (unpublished). If the left side of the funnel (where negative small studies would appear) has fewer points than the right side, it suggests that negative findings were not published, inflating the apparent effectiveness of the treatment.

Statistical tests for funnel plot asymmetry: Egger's test and Begg's test can formally assess whether asymmetry is present.

Red Flags in Studies

Watch for these warning signs when evaluating peptide research:

Study Design Red Flags

No control group or inadequate control (comparison to historical data rather than concurrent control)
No blinding for subjective outcomes
Very small sample sizes with strong conclusions
No power calculation or sample size justification
Primary endpoint changed after the study began (without clear justification)
Per-protocol analysis presented as the primary analysis without ITT

Statistical Red Flags

P-values reported as "less than 0.05" rather than exact values
Many outcomes tested without correction for multiple comparisons
Reporting only relative risk reductions without absolute numbers
Confidence intervals not reported
Post-hoc subgroup analyses presented as main findings
Statistical methods inappropriate for the data type

Reporting Red Flags

Abstract conclusions do not match the actual results
Selective reporting of only positive outcomes
Discrepancy between registered protocol (on ClinicalTrials.gov) and published results
Important limitations not discussed
Overly enthusiastic language ("groundbreaking," "revolutionary," "miracle")

Source Red Flags

Published in a predatory journal (check Beall's list or Think.Check.Submit)
No peer review
All authors from a single institution, especially if that institution commercializes the product
Funded entirely by the company that sells the product, with no independent replication
Not indexed in PubMed or major databases

Predatory Journals

Predatory journals are publications that prioritize profit over academic rigor. They charge authors publication fees but provide minimal or no peer review. Their articles often appear in search results alongside legitimate research, making them difficult for non-experts to identify.

Warning signs of predatory journals:

Aggressive email solicitation for manuscript submissions
Very fast turnaround from submission to publication (days rather than months)
No recognizable editorial board (or a board with members who are unaware they are listed)
No impact factor, or a fake impact factor from a non-recognized indexing service
Vague or absent peer review process
Grammatical errors in the journal's own website

How to check: Use resources like Think.Check.Submit (thinkchecksubmit.org), check if the journal is indexed in PubMed or the Directory of Open Access Journals (DOAJ), and look for it in the Journal Citation Reports for impact factor data.

Practical Checklist for Evaluating a Peptide Study

Use this checklist when you encounter a study cited in support of a peptide claim:

What type of study is it? In vitro, animal, or human? If animal, how relevant is the model?
Is there a control group? What was the control (placebo, active comparator, nothing)?
Was the study randomized and blinded? If not, why not, and how might this affect the results?
How many subjects/animals were included? Was a power calculation performed?
What were the primary endpoints? Were they predefined and clinically meaningful?
What are the actual effect sizes? Not just p-values, but the magnitude of the effect.
Are confidence intervals reported? How wide are they?
Who funded the study? Are there conflicts of interest?
Where was it published? Is it a reputable, peer-reviewed journal?
Has the finding been replicated? By independent groups in different settings?
Does the conclusion match the data? Or does the abstract overstate the findings?
If animal data, has it been confirmed in humans? If not, this is hypothesis-generating only.