Large or Slow Uploads should not Terminate Mysteriously
I live on the other side of the world (for most folks reading this), and my home internet connection takes a while to move data around.
I was trying to upload a 170Mb data file (400,000 text records compressed to .gz format), but found that the server side kills the connection before even a few Mb get across. I have tried many times, and the behavior seems almost precisely repeatable.
I don't get the same behavior with smaller files. The Databricks community edition too does not have any problems uploading the same 170Mb file.
Am I missing something? Thanks for any hints!
5
votes
