A problem with the current json based checkpoint files is that this results in a lot of relativley small text files. There are standardized binary file formats as Parquet which might be a good alternative