If you cant hold the entire prediction
set in RAM, you are predicting too far ahead.
not to purge anything in the current prediction set.
purge old data in groups, not one thing at a time, it reduces memory
Keep track of how much data is waiting in the queue to be
loaded, and purge that much.