Let’s discuss Code LLMs, their benchmark, downsides, and whether they can replace software engineers.