Once more, this proved to be quite difficult to do on Windows, so I'll explain how I finally got it to work. I'll assume you've read the previous post about the Theano installation and that you have mingw (64 bits) installed. The order in which you do these steps shouldn't really matter, but just in case, you can follow my order.
First, you need to download Open BLAS. Get the one called:
OpenBLAS-v0.2.14-Win64-int32.zip (I'm not sure if this also works with the -int64 version)
Next, make a folder C:\openblas. I don't think it must have this exact name, but it must not contain any spaces, so keep it simple.
Now, add this to your .theanorc file
Make sure you type it exactly like that, with the -L and everything.
Next, from the OpenBLAS zip you downloaded, extract the libopenblas.dll file from the bin folder and the libopenblas.a file from the lib folder to the openblas folder you created. I'm not sure if you need both or not, the dll might be enough, but I just copied both.
Finally, you also need to make the dependencies of this library available. I'm sure there's a better way to do this, but what I did was copy everything from C:\Program Files\mingw-w64\x86_64-5.1.0-posix-seh-rt_v4-rev0\mingw64\x86_64-w64-mingw32\lib (where I installed my mingw 64 bits) to the openblas folder I created. I guess you could also add this to your system Path, but that's arguably even uglier.
This did it for me. Please comment if you have any questions or suggestions on how to simplify this process.