Building a Social Media AI Agent with Smolagents

A light-weight library to build AI agents — great starting point to build and work with agents

Smolagents is a library for building AI agents from Hugging Face. Compared to other AI agent frameworks, it's lightweight, and allows agents to write actions in code.

I like the simplicity of the library - perfect for getting started creating agents (AIs that can take actions). Documentation is easy to understand and comes with plenty of bite-sized examples. I also like how it integrates with Hugging Face, meaning you can have your agents use publicly available Spaces as tools from the hugely popular platform.

To test this out, I built a social media agent that

1. comes up with an image prompt with a pre-defined theme (in this case, I wanted pictures of dogs on the beach)

2. uses a text-to-image tool to generate an image

3. comes up with a caption for social media posting

4. uses a custom tool to post to Bluesky

Here is my setup:

• AI Model: a locally hosted Qwen2.5(I am using the 14b coder tool model), using Ollama (quick how-to guide)

• Tools:

  • An image Generation Tool from Hugging Face Spaces: https://huggingface.co/black-forest-labs/FLUX.1-schnell

  • a custom tool the AI agent can use to post to Bluesky

Code

Here’s the code for creating the agent. As you can see, it is very straightforward and easy to understand.

# define the LLM model used for the agent
model = OpenAIServerModel(
    model_id="hhao/qwen2.5-coder-tools:14b",
    api_base="http://localhost:11434/v1",
    api_key="ollama",
)

# text to image tool directly from Hugging Face
image_generation_tool = Tool.from_space(
    "black-forest-labs/FLUX.1-schnell",
    name="image_generator",
    description="Generate an image from a prompt"
)

# tool to post to Bluesky
bluesky_image_poster_tool = BlueSkyImagePoster()

# creating the agent
agent = CodeAgent(
    tools=[image_generation_tool, bluesky_image_poster_tool], 
    model=model, 
    add_base_tools=True
)

# running the agent
result = agent.run(
    """
    Could you generate an image of a sunny beach with dogs in it? Include between 1 to 5 dogs. Feel free to include scenaries or landscapes or other objects/animals or people. use detailed and specific prompt for the FLUX model. Use industry best practices for the prompt. High-end photography, award-winning photos, still life photography, and/or cinematic photography. choose between different photography styles, lighting, weather, seasons, and time of the day in the images. 
    
    Once the image is generated, generate a fun, engaging social media caption for this dog beach image and post it to Bluesky.
    
    Consider the following guidelines:
    - Use emotive and playful language
    - Include relevant hashtags, always include #dogsonthebeach #petlife #dogsofbluesky
    - Keep it concise (under 280 characters)
    - Capture the joy and energy of dogs on a beach
    """
)

Notes and Learnings

• For the AI model, use one that can read/write code. This gives me the best result, which makes sense as I am using code agents from the library (CodeAgent class) which need to write actions in code.

  • Note that Smolagents also supports the standard way of writing actions in JSON with the ToolCallingAgent class.

• Agents CAN fail to execute the task sometimes. Unlike programming directly, which gives you predictable and deterministic outcome, working with agents can be unpredictable. How the agent decides to use the tools will vary, and the LLM model used for the agent will have an impact.

• On the official documentation, to use model on Ollama you'd go through LiteLLM. Alternatively you can use the OpenAIServerModel class as below:

from smolagents import OpenAIServerModel

model = OpenAIServerModel(
	model_id="hhao/qwen2.5-coder-tools:14b",
	api_base="http://localhost:11434/v1",
	api_key="ollama",
)

• When creating a tool, note that "output_type" supports most built-in types but not all of them. It didn't support dict which was what I was originally using

• I published my Bluesky posting tool to a Hugging Face space. Although in the documentation it should be pretty straightforward, I did struggle for a couple of hours to make it work - there are validation checks that the tool need to pass (rightfully so) and other restrictions and I had to make changes to the tool. I have published the tool to https://huggingface.co/spaces/thebrandonwu/bluesky-poster but will need to clean it up before making it public for others to use. (also waiting for this issue to get resolved)

Thoughts

Smolagents impressed me with its balance of simplicity and flexibility, making it an excellent choice for building AI agents, especially for beginners or iterative projects. Below are some specific insights from my experience:

I was able to build an agent and a custom tool with smolagents relatively quickly with my intermediary level of Python knowledge. It is certainly a library I'd continue to build with for the easy of use, and therefore the frequency I can iterate my project with it. I think it is a brilliant tool for anyone, especially those just starting to build with AI agents, to start building with.

Although it isn't necessary to have an agentic setup for this particular use-case — you could have easily generate an image prompt by, feed it into an image model, then generate a caption and post to a social platform via its SDK, using an agent is a fundamentally different setup and approach.

You write the tools an agent can decide to use in order to accomplish a task. The agent has, well, agency to decide how the task is accomplished. The ability to make decisions is a valuable difference. For instance, when generating and posting images, the agent can autonomously select the most relevant tools and handled the sequence of operations, reducing the need for direct intervention. This decision-making capability allowed it to adapt dynamically and perform tasks more efficiently, mimicking a human's ability to reason and prioritize. You could add more tools to its arsenal for it to do more down the line. If I were to continue working on this particular agent, I might add memory to it so that it remembers what had been posted before, visual recognition models for it to analyse image outputs before deciding to post, or have it generate content based on the performances of the posts.