File size estimates

[FrustratedWithTIP]

Have anyone built a complete json file for a single plan. What were the experiences - run time? File size ? Tech used ? and how are the files stored , cloud / local storage .

From our tech expert opinion , all the producer of these files should be running into problem of long running extract jobs, no database space, file size issues and/ or cannot produce the file after certain run time has elapsed. Would like to know everyone's experience and get an estimate of how everyone is making progress so far.

[tparency]

It may be helpful to understand your tech stack and what you experienced. Is the tech expert running into this or is this from a limited run?

[FrustratedWithTIP]

Hey, we are looking at two options one is using ETL tool and one is using legacy system. Both the options are failing either the process are running longer or DB virtual space gets exhausted . Overall when running smaller subset of files everything works great , at least there is an output. But when we try to build a file for one plan we are not able to build a file yet.

[BobAtMedcost]

We have the same concerns and have not been able to do a full run due to storage and processing limitations. We have had to work with small subsets of data to do proof of concept testing. See the thread I created below.

Machine Readable File Size Expectations · Discussion #350 · CMSgov/price-transparency-guide (github.com)

[FrustratedWithTIP]

I agree. I haven't seen anyone on discussion forum that have been able to do a complete run of at least one single plan file.

[BobAtMedcost]

I see the recent schema change adding modifier code. While this is necessary for accurate pricing, we estimate it will increase the data required 4 to 5 fold.

[FrustratedWithTIP]

That's will make your data quite large. It will be a problem converting it to a json with all the hios and provider /service mapping joins.