All we need now is affordable hardware with the memory to store that much context.
post
replies:
Fantastic news. My only gripe with 3.8 is that 262k is very limiting with all the reasoning the model does.
Pi ends up compacting history 3-4 times for larger tasks before it finishes.
So they didn't even start training it?
replies:
permalink
fedilink
source
parent
hideshow 3 child comments
They most likely used a local fork for a while.
replies:
permalink
fedilink
source
parent
hideshow 2 child comments
If that were the case wouldn't they just merge the working local fork rather than merging and reviewing changes one by one?
all 10 comments