>>200351

personally, I suspect the lack of continual learning is the real limiting factor
the weights are static, any additional knowledge has to fit in the context and the amount of dependencies the system has to process constantly increases with context length
you need a medium to long term memory system that isn't just pure context management