Pretty much as topic says, Are SSL(HTTPS) file downloads slower than normal HTTP? If so is there anything I can do to speed them up, or any sort of compression or something I should have enabled in NGINX ?
Unless the data is highly compressible (e.g. text files), SSL won't significantly increase the traffic transmitted. If it is compressible, then you can enable gzip compression over SSL, but this has some security issues and isn't recommended unless absolutely needed.
It's more likely that the computation overhead of cryptographic operations will impact performance (although only if the client has a high-throughput connection, and also only for large files). If CPU is the bottleneck, then compression won't help; in fact, it'll make it worse since compression also has high computation overhead compared to just dumping the stream to file. There's not much tuning you can do on the client-side unless you have access to the client. On the server, you can make sure that the AES CPU feature is enabled.
Obviously the first thing to do is to test download over HTTP and then over HTTPS and see if there is even any performance impact from SSL, and if there is an impact, to see what the bottleneck is.
Some ISPs are known to throttle encrypted traffic so, depending on the ISP, SSL downloads can be significantly slower. When I was using Comcast almost 3 years ago an HTTPS connection would be about 1/10th of the speed of HTTP (this was somewhat resolved when they upgraded everybody to 100Mbps).