Creating marketing campaigns is often a complex and time-consuming process. Businesses aim to create real-time campaigns that are highly relevant to customer needs and personalized to maximize sales. Doing so requires real-time data analysis, segmentation, and the ability to rapidly create and execute campaigns. Achieving this high level of agility and personalization gives businesses a significant competitive advantage.
Successful marketing campaigns have always hinged on creativity and data-driven insights. Generative AI is now amplifying both of these elements, and advancements in generative AI have the potential to revolutionize the creation of marketing campaigns. By leveraging real-time data, generative AI generates personalized content such as targeted emails, social media ads or website content that’s targeted to the situation and available images, all at scale. This is in contrast to the current state of affairs, where marketers are constrained by manual processes and have limited creative resources at their disposal.
While traditional marketing methods have their place, the sheer volume of content needed in today’s landscape demands a smarter approach. Generative AI can help marketing teams launch campaigns quickly, efficiently, and with a level of personalization that was previously impossible, leading to increased engagement, conversions, and customer satisfaction.
In this blog, Google will go through the various steps of how data and marketing teams can harness the power of multimodal large language models (LLMs) in BigQuery to create and launch more effective and intelligent marketing campaigns. For this demonstration, we reference Data Beans, a fictional technology company that provides a SaaS platform built on BigQuery to coffee sellers. Data Beans leverages BigQuery’s integration with Vertex AI to access Google’s AI models like Gemini Pro 1.0 and Gemini Vision Pro 1.0 to accelerate creative workflows while delivering customized campaigns at scale.
Demonstration overview
This demonstration highlights three steps of Data Beans’ marketing launch process that leverages Gemini models to create visually appealing, localized marketing campaigns for selected coffee menu items. First, we use Gemini models to brainstorm and generate high-quality images from the chosen menu item, ensuring the images accurately reflect the original coffee items. Next, we use those same models to craft tailored marketing text for each city in their native language. Finally, this text is integrated into styled HTML email templates, and the entire campaign is then stored in BigQuery for analysis and tracking.
Step 1 and 1.1: Craft the prompt and create an image
We start the marketing campaign by creating the initial image prompt and the associated image using Imagen 2. This generates a fairly basic image that might not be totally relevant, as we have not supplied all the necessary information to the prompt at this stage.
picture_description = "Mocha Pancakes with Vanilla Bean ice cream"
llm_human_image_prompt = f"Create an image that contains a {picture_description} that I can use for a marketing campaign."
human_prompt_filename = ImageGen(llm_human_image_prompt)
img = Image.open(human_prompt_filename)
llm_human_image_json = {
"gcs_storage_bucket" : gcs_storage_bucket,
"gcs_storage_path" : gcs_storage_path,
"llm_image_filename" : human_prompt_filename
}
img.thumbnail([500,500])
IPython.display.display(img)
Step 1.2. Enhance the prompt
Once we create the initial image we now focus on improving our image by creating a better prompt. To do this we use Gemini Pro 1.0 to improve on our earlier Imagen 2 prompt.
example_return_json = '[ { "prompt" : "response 1" }, { "prompt" : "response 2" }, { "prompt" : "A prompt for good LLMs" }, { "prompt" : "Generate an image that is awesome" }]'
llm_generated_image_prompt=f"""For the below prompt, rewrite it in 10 different ways so I get the most creative images to be generated.
Try creative thinking, generate innovative and out-of-the-box ideas to solve the problem.
Explore unconventional solutions, thinking beyond traditional boundaries, and encouraging imagination and originality.
Embrace unconventional ideas and mutate the prompt in a way that surprises and inspires unique variations.
Think outside the box and develop a mutator prompt that encourages unconventional approaches and fresh perspectives.
Return the results in JSON with no special characters or formatting.
Limit each json result to 256 characters.
Example Return Data:
{example_return_json}
Prompt: "Create a picture of a {picture_description} to be used for a marketing campaign."
"""
llm_success = False
temperature=.8
while llm_success == False:
try:
llm_response = GeminiProLLM(llm_generated_image_prompt, temperature=temperature, topP=.8, topK = 40)
llm_success = True
except:
# Reduce the temperature for more accurate generation
temperature = temperature - .05
print("Regenerating...")
print(llm_response)
Step 1.3. Generate images from LLM-generated prompts
Now that we have enhanced prompts, we will use these to generate images. Effectively, we are using LLMs to generate prompts to feed into LLMs to generate images.
llm_json = json.loads(llm_response)
# Add an image to the generation that will not contain any food items. We will test this later.
llm_json.append({'prompt' : 'Draw a coffee truck with disco lights.'})
image_files = []
for item in llm_json:
print(item["prompt"])
try:
image_file = ImageGen(item["prompt"])
image_files.append ({
"llm_prompt" : item["prompt"],
"gcs_storage_bucket" : gcs_storage_bucket,
"gcs_storage_path" : gcs_storage_path,
"llm_image_filename" : image_file,
"llm_validated_by_llm" : False
})
except:
print("Image failed to generate.")
Steps 2-3. Verify images and perform quality control
We are now going to use LLMs to verify the output that was generated. Effectively, we ask the LLM to check if each of the generated images contains the food items we asked for. For example, does the image contain the coffee or food items from our prompt? This will help us not just to verify if there is something abstract, but will also help us to verify the quality of the image. In addition, we can check if the image is visually appealing — a must for a marketing campaign.
example_return_json = '{ "response" : true, "explanation" : "Reason why..." }'
llm_validated_image_json = []
number_of_valid_images = 0
llm_validated_image_prompt = f"""Is attached image a picture of "{picture_description}"?
Respond with a boolean of true or false and place in the "response" field.
Explain your reasoning for each image and place in the "explanation" field.
Return the results in JSON with no special characters or formatting.
Place the results in the following JSON structure:
{example_return_json}
"""
for item in image_files:
print(f"LLM Prompt : {item['llm_prompt']}")
print(f"LLM Filename : {item['llm_image_filename']}")
imageBase64 = convert_png_to_base64(item['llm_image_filename'])
llm_success = False
temperature = .4
while llm_success == False:
try:
llm_response = GeminiProVisionLLM(llm_validated_image_prompt, imageBase64, temperature=temperature)
print(f"llm_response : {llm_response}")
llm_json = json.loads(llm_response)
llm_success = True
except:
# Reduce the temperature for more accurate generation
temperature = temperature - .05
print("Regenerating...")
# Mark this item as useable
if llm_json["response"] == True:
item["llm_validated_by_llm"] = True
number_of_valid_images = number_of_valid_images + 1
item["llm_validated_by_llm_explanation"] = llm_json["explanation"]
llm_validated_image_json.append( {
"llm_prompt" : item["llm_prompt"],
"gcs_storage_bucket" : item["gcs_storage_bucket"],
"gcs_storage_path" : item["gcs_storage_path"],
"llm_image_filename" : item["llm_image_filename"],
"llm_validated_by_llm" : item["llm_validated_by_llm"],
"llm_validated_by_llm_explanation" : item["llm_validated_by_llm_explanation"]
})
Step 4. Rank images
Now that we have gone through verification and quality control, we can choose the best image for our needs. Through clever prompting, we can use Gemini Pro 1.0 again to do this for us for the thousands of images we generated. To do this, we ask Gemini to rank each image based on its visual impact, clarity of message, and relevance to our Data Beans brand. We will then select the one with the highest score.
image_prompt = []
for item in image_files:
if item["llm_validated_by_llm"] == True:
print(f"Adding image {item['llm_image_filename']} to taste test.")
imageBase64 = convert_png_to_base64(item['llm_image_filename'])
new_item = {
"llm_image_filename" : item['llm_image_filename'],
"llm_image_base64" : f"{imageBase64}"
}
image_prompt.append(new_item)
example_return_json='[ {"image_name" : "name", "rating" : 10, "explanation": ""}]'
llm_taste_test_image_prompt=f"""You are going to be presented with {number_of_valid_images}.
You must critique each image and assign it a score from 1 to 100.
You should compare the images to one another.
You should evaluate each image multiple times.
Score each image based upon the following:
- Appetizing images should get a high rating.
- Realistic images should get a high rating.
- Thought provoking images should get a high rating.
- Plastic looking images should get a low rating.
- Abstract images should get a low rating.
Think the rating through step by step for each image.
Place the result of the scoring process in the "rating" field.
Return the image name and place in the "image_name" field.
Explain your reasoning and place in the "explanation" field in less than 20 words.
Place the results in the following JSON structure:
{example_return_json}
"""
llm_success = False
temperature=1
while llm_success == False:
try:
llm_response = GeminiProVisionMultipleFileLLM(llm_taste_test_image_prompt, image_prompt, temperature = temperature)
llm_success = True
except:
# Reduce the temperature for more accurate generation
temperature = temperature - .05
print("Regenerating...")
print(f"llm_response : {llm_response}")
Step 5. Generate campaign text
Now that we have selected the best image, let’s generate the best marketing text. We are storing all the generated data in BigQuery, so we generate text in JSON format. We are generating text to incorporate promotions and other relevant items.
example_return_json_1 = '{ "city" : "New York City", "subject" : "email subject" , "body" : "email body" }'
example_return_json_2 = '{ "city" : "London", "subject" : "marketing subject" , "body" : "The body of the email message" }'
imageBase64 = convert_png_to_base64(highest_rated_llm_image_filename)
llm_marketing_campaign = []
# Loop for each city
for city_index in range(0, 4):
print(f"Processing city: {city_names[city_index]}")
prompt=f"""You run a fleet of coffee trucks in {city_names[city_index]}.
We need to craft a compelling email marketing campaign for the below image.
Embrace unconventional ideas and thinking that surprises and inspires unique variations.
We want to offer exclusive discounts or early access to drive sales.
Look at the image and come up with a savory message.
The image contains "{picture_description}".
The marketing campaign should be personalized for {city_names[city_index]}.
Do not mention the weather in the message.
Write the response using the language "{city_languages[city_index]}".
The campaign should be less than 1500 characters.
The email message should be formatted in text, not HTML.
Sign the email message "Sincerely, Data Beans".
Mention that people can find the closest coffee truck location by using our mobile app.
Return the results in JSON with no special characters or formatting.
Double check for special characters especially for Japanese.
Place the value of "{city_names[city_index]}" in the "city" field of the JSON response.
Example Return Data:
{example_return_json_1}
{example_return_json_2}
Image of items we want to sell:
"""
llm_success = False
temperature = .8
while llm_success == False:
try:
llm_response = GeminiProVisionLLM(prompt, imageBase64, temperature=.8, topP=.9, topK = 40)
llm_json = json.loads(llm_response)
print(llm_response)
llm_success = True
except:
# Reduce the temperature for more accurate generation
temperature = temperature - .05
print("Regenerating...")
llm_marketing_campaign.append(llm_json)
llm_marketing_prompt.append(prompt)
llm_marketing_prompt_text.append(llm_response)
In addition, notice how we can use Gemini Pro 1.0 to localize the marketing messages for different countries’ native languages.
Step 6. Create an HTML email campaign
The generated artifacts are displayed as part of a web application. To simplify distribution, we need to to create an HTML email with embedded formatting as well as embedding our image. Again, we use Gemini Pro 1.0 to author our marketing text as HTML based on the images and text we created in the previous steps.
imageBase64 = convert_png_to_base64(highest_rated_llm_image_filename)
for item in llm_marketing_campaign:
print (f'City: {item["city"]}')
print (f'Subject: {item["subject"]}')
prompt=f"""Convert the below text into a well-structured HTML document.
Refrain from using <h1>, <h2>, <h3>, and <h4> tags.
Create inline styles.
Make the styles fun, eye-catching and exciting.
Use "Helvetica Neue" as the font.
All text should be left aligned.
Avoid fonts larger than 16 pixels.
Do not change the language. Keep the text in the native language.
Include the following image in the html:
- The image is located at: https://REPLACE_ME
- The image url should have a "width" of 500 and "height" of 500.
Double check that you did not use any <h1>, <h2>, <h3>, or <h4> tags.
Text:
{item["body"]}
"""
llm_success = False
temperature = .5
while llm_success == False:
try:
llm_response = GeminiProLLM(prompt, temperature=temperature, topP=.5, topK = 20)
if llm_response.startswith("html"):
llm_response = llm_response[4:] # incase the response is formatted like markdown
# Replace the image with an inline image (this avoids a SignedURL or public access to GCS bucket)
html_text = llm_response.replace("https://REPLACE_ME", f"data:image/png;base64, {imageBase64}")
item["html"] = html_text
#print (f'Html: {item["html"]}')
filename= str(item["city"]).replace(" ","-") + ".html"
with open(filename, "w", encoding='utf8') as f:
f.write(item["html"])
print ("")
llm_success = True
llm_marketing_prompt_html.append(prompt)
llm_marketing_prompt_html_text.append(html_text)
except:
# Reduce the temperature for more accurate generation
temperature = temperature - .05
print("Regenerating...")
Conclusions and resources
The integration of LLMs into creative workflows is revolutionizing creative fields. By brainstorming and generating numerous pieces of content, LLMs provide creators with a variety of quality images for their campaigns, speed up text generation that’s automatically localized, and analyze large amounts of data. Moreover, AI-powered quality checks ensure that generated content meets desired standards.
While LLMs’ creativity can sometimes produce irrelevant images, Gemini Pro Vision 1.0 “taste test” functionality lets you choose the most appealing results. Additionally, Gemini Pro Vision 1.0 provides insightful explanations of its decision-making process. Gemini Pro 1.0 expands audience engagement by generating content in local languages, and with support for code generation, eliminates the need to know HTML.
To experiment with the capabilities showcased in this demonstration, please see the complete Colab Enterprise notebook source code. To learn more about these new features, check out the documentation. You can also use this tutorial to apply Google’s best-in-class AI models to your data, deploy models, and operationalize ML workflows without ever having to move your data from BigQuery. Finally, check out this demonstration on how to build an end-to-end data analytics and AI application directly from BigQuery while harnessing the potential of advanced models such as Gemini. As a bonus, you’ll get a behind-the-scenes take on how we made the demo.