It would be interesting to read the post mortem (if they will release). In particular in which phase of the training it escaped the sandbox, in which way it used the information. Training is different from inference, I doubt there is a context there and where would the model store these information? In the weights?! That makes little sense to me

 

As opposed to the northern/southern hemisphere that has:

  • a clear reference point (equators)
  • traditionally we don’t model a rotation north/south

Western/eastern hemisphere

  • doesn’t have a clear reference (Greenwich?)
  • traditionally we model rotation on the north/south axis so western/eastern would depend from the time