2 options
Parse in CLI using tools like JQ and then take it to Python for further processing.
Parse using PySpark (community data bricks gives you free space)
↧
Answer by Ganesh Chandrasekaran for How to parse a large JSON file efficiently in Python?
↧