StefanKarpinski 1 hour ago [-]
In TensorFlow one doesn't write Python code and run it on a G/TPU, one writes Python code that uses an API to construct a TensorFlow graph computation which is compiled and runs on the G/TPU. It's really a completely separate programming language with different semantics and a totally separate runtime hiding behind a Python API. This is why you can't use any of Python's normal libraries, you have to use TensorFlow libraries.
"Julia runs on GPUs" actually means that you write Julia code and it is compiled to natively run on GPUs [1]. Similarly, this post is about writing Julia code and compiling it to run natively on TPUs, not calling some predefined TPU library. Yes, of course you _can_ call libraries that are compiled for GPUs and TPUs—as you can in Python or any other language—but in Julia, nearly arbitrary user code can be compiled and run on GPUs and TPUs. The major restriction on GPUs is that you cannot do dynamic allocation, so you need to write non-allocating Julia code, but that's common requirement for high-performance code anyway.
[1]
https://github.com/JuliaGPU/CUDAnative.jl