In recent weeks, the Obama Administration has been touting optimistic claims of large numbers of Americans who “have gained health coverage since several of the Affordable Care Act’s (ACA) provisions took effect.” A recent two-page report from the Department of Health and Human Services (HHS) claims that 16.4 million adults aged 18-64 gained coverage between 2010 and March 4, 2015. But how many of these are truly “newly insured,” and how many were already insured, or would have been insured, anyway?

The report claims that 2.3 million young adults, aged 19-25, are covered under the ACA provision allowing people to remain on a parent’s health plan (say, through an employer) until age 26. This provision took effect in September 2010, and the 2.3 million number includes only those covered before ACA exchange enrollment began in October 2013. The assumption appears to be that these 2.3 million young adults would all be uninsured without the ACA. However, this does not adequately characterize the pre-ACA health care landscape, in which many of these individuals were insured anyway. For one thing, many employers already covered the employees’ children until age 23 (even more did so if the children were enrolled in college). Many people in that age group were employed and covered under their own employer’s health plans, and others were covered under college-sponsored student health plans. How many of those 2.3 million young adults would have been insured anyway, without the ACA? The report doesn’t tell us, but it’s surely far more than zero.

The other 14.1 million, who are said to have gained coverage since October 2013, obtained coverage either from Medicaid (11.2 million, according to HHS), or through the ACA exchanges.  Again, we have to ask how many of these people would have been uninsured without the ACA. The implication of the report is “all of them” but that is clearly not the case.

For the non-Medicaid insured, the number covered by the ACA exchanges has to be offset by the number of people who previously had individually-purchased coverage. Some may have, and chose to replace it with ACA coverage; and a large number, perhaps the majority, enrolled in ACA coverage after their previous coverage was canceled because it did not meet ACA criteria. The number of plan cancellations is highly controversial, with estimates ranging from 2.3 million to 6.7 million plans canceled initially. Some of these cancellations were temporarily reversed by the administration’s decision to postpone enforcement of the cancellation requirements – a decision that was accepted by only some states. Naturally, the number of plan cancellations understates the number of people who lost coverage, since some cancellations were for family coverage.

While it is difficult to pin down a precise estimate of the number of people who lost coverage due to ACA-mandated plan cancellations, the number of people covered by exchange plans is much higher than the number of “newly insured.” It is even possible that the number of people who lost coverage exceeds the number who gained coverage; and presumably not all those who lost their health coverage are currently insured.

For the Medicaid expansion – which is the largest category of potential “newly insured” – one has to make similar offsets. Under the ACA those eligible for Medicaid are not permitted to buy ACA exchange coverage, even if they wish to and are able to pay for it. (They may be able to buy insurance directly from an insurer off the exchange, but would not be eligible for income-based subsidies.) This means that some of those who had their pre-ACA insurance canceled may have enrolled in Medicaid – especially in states that expanded Medicaid eligibility. Indeed, one of the most widely publicized early enrollees was a 30-year-old law student who dropped his university-sponsored student health plan and enrolled in Medicaid through an exchange website.  Obviously, not every new Medicaid enrollee is a law student, and no doubt some did not have any prior coverage – but some did.

Indeed, even in states that did not implement the ACA Medicaid expansion, there were gains in Medicaid enrollment due to what is known as the “woodwork effect.” This is a term for what occurs when publicity about a benefit causes more people to sign up for that benefit, even if they could have signed up before. Perhaps they were previously eligible and didn’t realize it, or perhaps they were motivated by publicity. In the case of the ACA, the individual mandate penalty provides additional incentive to seek coverage. Uninsured people who are Medicaid-eligible have to pay a penalty for being uninsured – unless they enroll in Medicaid. This accounts for a significant fraction of new enrollees.

It’s useful to note that in some sense, people eligible for Medicaid but not actually enrolled are de facto insured. Unlike private insurance, Medicaid coverage is retroactive. A Medicaid-eligible individual – or a health care provider treating one – can file claims for Medicaid payment for services provided up to 90 days prior to that individual’s enrollment in Medicaid (provided that the individual was Medicaid-eligible at the time of service). This means that, in effect, unenrolled Medicaid-eligible individuals are covered in case they need treatment, even without being enrolled. (Hospitals are very good at getting Medicaid-eligible patients enrolled, so they can be reimbursed for services.) Therefore, those who were previously eligible for Medicaid and signed up to avoid the penalty for being uninsured, or as a result of widespread publicity, are only “newly insured” in a technical sense.

One approach to get around these problems of counting new enrollments in the presence of cancellations is to look at survey data, and the administration cites survey data as well. However, this just exchanges one set of problems for another, because survey data relating to health coverage is notoriously unreliable.  Census Bureau surveys have found that as much as 43 percent of those enrolled in Medicaid report that they are not so enrolled, and an additional 17 percent report that they are “uninsured” even when they are enrolled in Medicaid. This is known as the “Medicaid undercount,” and has been the subject of much research. However, this research is possible only because the Census Bureau, as a government agency, can cross-match survey data with enrollment data in Medicaid, a government program.

No such cross-check exists for surveys of those with private-sector coverage. The most widely-cited survey-based estimates of health coverage (both cited by HHS), are the Gallup-Healthways Well-Being Index survey, which has no way to verify coverage at all, and the Census Bureau’s Current Population Survey, which changed the way it collected health insurance data in 2013, making pre- and post-ACA data not directly comparable.

The bottom line is, while the administration claims that 16.4 million Americans are “newly insured” and that the percentage who are uninsured is dropping steadily, this is really a “best case” estimate, since it counts many people who are insured in new programs but were also insured before. The true number of newly insured is surely much lower, and it is even possible that the net change in insurance is actually negative. Without more detailed data, which may exist but has not been publicly released, the true impact of the ACA on overall health insurance coverage is unknown.