Start with the research question—not the AI label
A claim that “AI improves housing decisions” is too vague to evaluate. Improve which decision? For whom? Compared with what? Measured over what period and in what place?
Housing-related AI can involve generative tools, predictive analytics, computer vision, home searches, or price estimates. The National Association of REALTORS® describes these as current real estate use cases, while also identifying concerns about data bias, privacy, inconsistent rules, and the continuing role of human professionals. Its overview is useful context, but it does not establish that a particular model is accurate or fair. NAR, “Artificial Intelligence (AI) in Real Estate” (opens in a new tab)
Before treating a result as evidence, write the claim in a form you could test:
Among [defined population] in [defined geography], does [specified AI system or workflow], compared with [specified alternative], change [measurable outcome] over [specified period]?
That sentence forces several missing details into the open. A vendor may say its tool “supports better valuation.” A study should tell you what “better” means, who used the tool, what comparison was made, and how the result was measured.
Six questions for inspecting housing AI research
1. What exactly is the research question?
Look for a narrow question rather than a broad promise. “Can AI help plan affordable housing?” could refer to design speed, construction cost, unit count, accessibility, resident satisfaction, approval time, or something else entirely. Those outcomes aren’t interchangeable.
Also check whether the paper studies a real deployed system, a prototype, a simulation, or people’s opinions about AI. A simulated layout study cannot by itself establish what happens to residents, costs, rents, or approvals in actual projects.
2. What is the dataset, and where did it come from?
Ask for the data’s source, collection dates, labels, missing values, exclusions, and ownership. If people labeled the data, who were they and what instructions did they receive? If the dataset was inherited from another project, can the researchers trace its lineage?
A public comment record published by the U.S. National Institute of Standards and Technology (NIST) on 2021-08-12, concerning a proposed framework for identifying and managing AI bias in the United States, raises these questions directly. Commenters discuss whether data is sufficiently representative, how labels and data lineage should be documented, and how bias should be identified, measured, and managed. This is a comment record—not a completed housing study—so it supplies an evaluation lens, not evidence of a particular housing effect. NIST, “Comments Received on A Proposal for Identifying and Managing Bias in Artificial Intelligence” (opens in a new tab)
3. What geography and population does the evidence cover?
A model tested on listings in one metropolitan area may not perform the same way in rural counties, different states, or countries with different housing rules and data practices. Record the geography, property types, market conditions, and population represented.
Be precise about whose experience is included. A study of real estate professionals’ technology use is not a study of renters’ housing access. A study of single-family price estimates is not evidence about affordable-rental development.
4. What is the outcome measure—and what is the denominator?
Separate an operational measure from a housing outcome. Faster drafting, more generated design options, or higher user satisfaction may be useful operational findings. They do not automatically demonstrate better valuations, lower housing costs, more affordable units, improved access, or fewer discriminatory outcomes.
For example, NAR’s 2025 REALTORS® Technology Report says it surveyed members about technology experience and use. The page reports that 23% of respondents used AI tools daily, 55% said AI had a positive effect on their business, and 75% of AI users cited listing-description writing as a use case. Those are survey results about respondents’ reported behavior and views—not tests showing that AI improved sale prices, valuation accuracy, affordability, or fair-housing outcomes. The retrieved page also does not provide the full sample design, geography, questionnaire, or subgroup results. NAR, “REALTORS® Technology Report” (opens in a new tab)
Check the denominator every time. “75% of AI users” differs from “75% of all respondents.” “Accuracy improved” requires a defined baseline, error measure, and evaluation sample.
5. Where does human review happen?
Find out whether people reviewed the inputs, model outputs, or final decisions. Ask who could override the system, what counted as an error, and whether reviewers received the same information for every case.
Human involvement is not a magic fairness stamp. A reviewer may rely heavily on an automated score, lack authority to change it, or review only unusual cases. NIST’s public comments emphasize human interventions, stakeholder engagement, traceability, and verification and validation as issues worth examining across the AI lifecycle. That guidance frames questions; it does not prove that human review corrected bias in any housing application. NIST, “Comments Received on A Proposal for Identifying and Managing Bias in Artificial Intelligence” (opens in a new tab)
6. What are the study’s bias limits?
A credible paper should name what it tested and what it could not test. Look for subgroup results, missing groups, unequal error rates, changes over time, and possible proxy variables. A model can perform well on an overall average while performing differently across neighborhoods, property types, or demographic groups.
Also distinguish observed disparity from its explanation. If one group receives different outputs, the study may identify a pattern without proving why it occurred. Possible explanations include data coverage, measurement choices, historical patterns, workflow decisions, or deployment conditions. The evidence needs to separate those possibilities rather than quietly turn correlation into a cause.
A labeled hypothetical comparison
The following is a fictional example for evaluating study design, not a finding about any real AI system.
| Feature | Study A: strong description | Study B: weak description |
|---|---|---|
| Question | Tests whether a named valuation model changes median absolute error against a stated baseline | Says the model produces “more accurate values” |
| Dataset | Identifies source, dates, property types, exclusions, and train/test split | Says it used “millions of housing records” |
| Geography | Names three counties and reports results separately | Says it works “across markets” |
| Outcome | Defines the error measure and gives the denominator | Uses “accuracy” without explaining the calculation |
| Human review | Explains who reviewed outputs and when overrides occurred | Says experts were “in the loop” |
| Bias limits | Reports subgroup results and missing data | Says the model is “unbiased” without a test or definition |
Study A may still be wrong. But it gives readers something to inspect, reproduce, or challenge. Study B may contain a useful product claim, yet its language alone cannot establish research evidence.
A five-minute source-check worksheet
Copy these prompts into your notes before sharing a housing AI claim:
- Question: What decision or task was tested?
- System: Which model, version, or workflow was used?
- Population and geography: Who and where are represented?
- Dataset: What are the source, dates, labels, exclusions, and denominator?
- Comparison: Compared with what human process, rule, or baseline?
- Outcome: What was measured, using which error or success definition?
- Human review: Who reviewed, overrode, or approved outputs?
- Bias analysis: Which groups and failure modes were examined?
- Limits: What does the study explicitly say it cannot establish?
- Provenance: Is this peer-reviewed research, a survey, a public comment, an industry overview, or a vendor claim?
If several boxes remain blank, narrow the conclusion. “This source describes a proposed method” is more accurate than “research proves the method works.” That distinction is especially important when a housing decision affects access, price, valuation, planning, or public resources.
Related reading
Keep up with real estate
Get the next useful update in your inbox.
Choose Daily Sellable or the Sunday Weekly Sellable edition. Each briefing explains the verified news, its real estate implications, and links to the source-backed article. Change frequency or unsubscribe from any issue.