ChatGPT和Claude 對比測試(以Review MLIR Codegen代碼為例)。

Claude在MLIR代碼分析上完全超越了ChatGPT並表現十分驚艷,請閱讀全文或者自己註冊感受它的強大。結論:在本文的任務中,Claude > ChatGPT >> NewBing

0x0. 前言

這裡將以oneflow IR部分中的一個Codegen任務《目標是在mlir codegen中支持oneflow stream,用oneflow stream替換pass中自己生成的stream,PR鏈接為:https://github.com/Oneflow-Inc/oneflow/pull/10149》為例,來對比一下newibing(chatgpt)和claude對mlir的理解能力。claude是Anthropic公司推出的類似於chatgpt的聊天機器人,這家公司是OpenAI的最大競爭對手之一,因為創辦這家公司的人也是OpenAI的前員工。然後Claude是參考這個issue: https://www.zhihu.com/question/594115372/answer/2988759047 將其直接添加到slack裡進行對話。

0x1. PR簡介

PR鏈接為:https://github.com/Oneflow-Inc/oneflow/pull/10149

這個PR實現了3個Pass (定義在 OneFlowPasses.td),也就是:

def EliminateAllocOpsPass : Pass<"eliminate-alloc-ops","ModuleOp"> { let summary = ""; let constructor = "mlir::oneflow::createEliminateAllocOpsPass()"; let dependentDialects = ["pdl_interp::PDLInterpDialect","pdl::PDLDialect"];}def AppendOneFlowStreamPass : Pass<"append-ofstream","ModuleOp"> { let summary = "append oneflow stream to gpu function arguments"; let constructor = "mlir::oneflow::createAppendOneFlowStreamPass()";}def MgpuToOneFlowStreamPass : Pass<"mgpu-to-ofstream","ModuleOp"> { let summary = "convert mlir abi about mgpu to oneflow stream,this pass should be invoked after append-ofstream pass"; let constructor = "mlir::oneflow::createMgpuToOneFlowStreamPass()";}