Ok, this post was written last Friday and scheduled for today and... it rained a lot, haha.
This weekend, the AI community rushed to validate Matt's model, and it's very unclear that the Reflection technique is so effective through a dataset and fine-tuning.
In any case, I still trust Matt, but worked or not, this technique is indeed useful on system or normal prompting, also the concept to find ways to let the models invest more computation on better rationales.
Confirmed: the anticipated superior performance of the Reflection model was incorrect. A postmortem is pending to investigate why higher evaluation scores were initially observed, but couldn't be replicated later. https://x.com/mattshumer_/status/1833619390098510039
Ok, this post was written last Friday and scheduled for today and... it rained a lot, haha.
This weekend, the AI community rushed to validate Matt's model, and it's very unclear that the Reflection technique is so effective through a dataset and fine-tuning.
In any case, I still trust Matt, but worked or not, this technique is indeed useful on system or normal prompting, also the concept to find ways to let the models invest more computation on better rationales.
Let's see what happens :popcorns:
Confirmed: the anticipated superior performance of the Reflection model was incorrect. A postmortem is pending to investigate why higher evaluation scores were initially observed, but couldn't be replicated later. https://x.com/mattshumer_/status/1833619390098510039
As a former phylosofy student I expected you to delve into the technical terms definition and then grounding it on your area of expertise.
If one day you try to delve into this theoretical jungle, I'll be the one bringing popcorn 😂
Congrats! You pass the AI detector as a human, thanks to the "philosophy" typo 🤪
omg
Although I did see something was amiss, couldn't tell what... And yet, it was so obvious. I need to sleep more