Raj Darshan Pachori

@kartik-nvj Thats a valid point. Successful-outcome is something which would build the credibility and not just the cost. Havings said that there's a merit in routing the low complexity tasks to the cheaper models if one is able to get the confidence in prior evaluation. And capability has to be understood through both prior eval score and refinement of model deployed to live.

Rishita Sharma Those are best practices. Some of that can be automated and some are guidelines for developers to be followed as part of their day to day work. Sorry, cannot get the eval question.

Agree Julian Neagu

Those are valid questions and answer to those can be more informative. OpenRouter who publised the trend didn't really provide the detailed level information.

Absolutely! Token cost is only one part of the equation. Engineering teams certainly need to observe other engineering parameters while integrating the open-weight models.

Cool projects!

Good article. Do you foresee any newer trends emerging in the field of Software Engineering?

AI pair programming tools undoubtedly hold significant potential for improving developer productivity. Numerous studies have highlighted the productivity gains offered by these tools, including a 2022 study by GitHub Copilot that reported a 55% improvement. However, other studies have found much lower productivity gains, ranging from 5% to 15%, with some even suggesting that Copilot may negatively impact productivity.

Local LLMs can really be handy for building small application automation requiring high data security standards.

Have been learning coding with CoPilot for last few months however didnt come across GitHub models so far. Learned something new today. Thank you.