Originally published on 2023-08-06 at juanreyero.com.
Historical note, September 2026: This is an experiment from August 2023, using the models and API available then. I have retained the original code and conclusions as a record of that work, rather than updated implementation guidance. The original experiment does not establish that the resulting data is fully anonymized.
Note added on 2025-07-28: It’s been two years of programming with and for LLMs. This article sounds so naïve now.
Say you made a website in which customers buy personalized gifts . Each gift comes with a message, written by the customer in whatever language the customer wants. The use of grammar, punctuation and capitalizations in the messages is often creative.
You would like to be able to offer reasonably normative messages to your customers. You would also like to store a fully anonymized version of the messages; replace all proper names with a [proper_name] placeholder, place names with [place_name], dates with [date], times with [time], and geographical coordinates with [coordinates].
Is it possible to use OpenAI’s gpt-3.5-turbo to do this?
Luckily you happen to have some tens of thousands of messages laying around, so you can actually test whatever solution you come up with.
The naïve approach: just ask the model to do it
The first thing I tried was the straight-forward option: just ask the genie to do it for you, with a system prompt and a user prompt like this:
system_prompt = ("You are a proof-reader. You will be given "
"a text that may be poorly written, perhaps "
"with bad punctuation and wrong capitalization. "
"Your task is to first correct the grammar, "
"punctuation, and capitalization of the text. "
"Once this is done you should anonymize it by "
"replacing all the person's proper names in the text "
"with [proper_name], place names with [place_name], "
"dates with [date], times with [time], and "
"geographical coordinates with [coordinates]")
user_prompt = f'Correct and anonymize the following text: {message}'
This doesn’t work with GPT-3.5. It sometimes produces the desired result, but it often misses the anonymization elements, and generally makes a mess of things.
Function calling
Function calling is a wonderful tool that allows you to get structured data from the model. You describe functions and their inputs in your call to the model; if the model decides that it wants to call one of the functions, it will come back with a request to call the function and, crucially, with a dictionary containing the function’s arguments.
And, most interesting for the problem at hand, you can also force the model to call a function. If you do, you will get the structured data you would need to call the function.
So what we do is ask the model to call a function that needs, as arguments, the corrected text and the anonymization elements. Once we get back the dictionary with the arguments for the function we can anonymize by simply searching and replacing the anonymization elements in the corrected text.
This actually works. Interestingly, the GPT-3.5 model is not able to replace the anonymization elements in the text by their placeholder, but it can reliably find them.
Sample code
This is the function I have been using to play with the OpenAI api:
import json
from typing import List, Dict
import openai
# Ensure that the openai key is loaded, with something like
Api_loaded = False
def ensure_api(api_key):
global Api_loaded
if not Api_loaded:
openai.api_key = api_key
Api_loaded = True
def chat_completion_request(messages: List[Dict],
api_key: str,
functions: List[Dict] = None,
call_function: str = None,
temperature: float = 1, # 0 to 2
model_name: str = 'gpt-3.5-turbo-0613',
attempts: int = 4) -> Dict:
args = {'model': model_name,
'messages': messages,
'temperature': temperature}
if functions:
args['functions'] = functions
if call_function:
if not functions:
raise RuntimeError(f'Cannot call function {call_function} '
'if no functions are defined')
found_function = len([fdesk for fdesk in functions
if fdesk['name'] == call_function])
if not found_function:
raise RuntimeError(f'Cannot call undefined function {call_function}')
if found_function > 1:
raise RuntimeError(f'Found more than one definitions of {call_function}')
args['function_call'] = {'name': call_function}
try:
ensure_api(api_key)
response = openai.ChatCompletion.create(**args)["choices"][0]["message"]
function_call = response.get('function_call', None)
if function_call:
try:
return {
'call_function': function_call['name'],
'with_arguments': json.loads(
function_call['arguments']),
'success': True}
except json.decoder.JSONDecodeError:
# The model may have failed to produce correct json
if attempts:
return chat_completion_request(
messages=messages,
api_key=api_key,
functions=functions,
call_function=call_function,
temperature=temperature/2,
model_name=model_name,
attempts=attempts-1)
return {'success': False}
return {'reply': response['content'],
'success': True}
except openai.InvalidRequestError as exc:
raise RuntimeError('Invalid request error from OpenAI') from exc
We define a store_correct_text function like this:
functions = [{
"name": "store_correct_text",
"description": "Store a corrected version of the input text",
"parameters": {
"type": "object",
"properties": {
"correct_text": {
"type": "string",
"description": "A corrected version the text"
},
"person_proper_names": {
"type": "string",
"description": ("A list of the proper names of "
"persons in the text separated "
"by semicolons")
},
"place_names": {
"type": "string",
"description": ("A list of the names of places in "
"the text separated by semicolons")
},
"dates": {
"type": "string",
"description": ("A list of the dates in the text "
"separated by semicolons")
},
"geographical_coordinates": {
"type": "string",
"description": ("A list of the dates in the text "
"separated by semicolons")
},
"times": {
"type": "string",
"description": ("A list of the times (hours and minutes) "
"in the text separated by semicolons")
}
},
"required": ["correct_text",
"person_proper_names",
"place_names",
"dates",
"geographical_coordinates",
"times"]
}
}]
The system and user prompts are:
system_prompt = ("You are a proof-reader. You will be "
"given a text that can be poorly written, "
"maybe with bad punctuation and wrong "
"capitalization. Your task is to first correct "
"the grammar, punctuation, and capitalization "
"of the text, and once this is done to find all "
"the person's proper names in the text. "
"Person's proper names are those that identify "
"people, like Mark or Anne Frank. You should also "
"find all the proper names of places in the text. "
"You should also find the dates. You should also "
"find geographical coordinates (latitude and "
"longitude). You should also find the times "
"mentioned in the text (hour and minute).")
user_prompt = ('Correct the following text, and make it into '
f'a template:\n\n{message}')
We can now call chat_completion_request:
result = chat_completion_request(
messages=[{'role': 'system',
'content': system_prompt},
{'role': 'user',
'content': user_prompt}],
api_key='your api key possibly loaded from .env',
functions=functions,
call_function'store_correct_text')
Finally, if things worked, simply replace the anonymization elements:
if result['success']:
final_text = result['with_arguments'].get('correct_text', '')
for argname in ('person_proper_names',
'place_names',
'dates',
'geographical_coordinates',
'times'):
names = result['with_arguments'].get(argname, '')
for name in names.split(';'):
if name.strip():
# The placeholder is the argument name
# minus the final s
final_text = final_text.replace(name.strip(),
f'[{argname[:-1]}]')

