Synthetic data + tool use for LLM improvements 🦙
00:00:00.000 | What I'm excited about is, we had this survey on augmented LLM a year ago, and all the idea
00:00:06.600 | is like, if you augment your LLM with something else, it can be a retriever, it can be search,
00:00:11.960 | it can be a tool, it can be a calculator, it can be a code execution, then you are not
00:00:17.560 | just distillating, like doing some data augmentation with your model, but you're actually adding
00:00:23.440 | some expert skills that possibly goes beyond the model weights.
00:00:28.200 | For instance, if your model can calculate something it was wrong before, and now it
00:00:34.520 | has access to a calculator, and you can retrain your model on that, then you're learning something
00:00:38.840 | new.
00:00:39.840 | If your model didn't know something about LLM 2, it probably doesn't know a lot about
00:00:43.480 | LLM 3, but now if it can search online about it, and then you train the model on that,
00:00:48.760 | Then you have a positive feedback loop like what we call expert iteration.