Forem: Jay Miller

Using Azure Storage Queues and Python to Automate Sending Tweets

Jay Miller — Thu, 03 Nov 2022 07:00:00 +0000

October was ADHD Awareness Month and this year for October, I wanted to do something to bring awareness and encouragement to folks with ADHD and other neurodiversities.

Jump To

Intro
Storage Queues
Creating Images
Twitter Access
Improving the Process
Running the Script
Was It Worth It
Check it Out

Aside from speaking at Refactr.Tech and appearing and a few podcasts, I wanted to create something that:

1. Would have a wider reach. 
2. Would let me play with Python and Azure

If I look at where I have the largest reach, that's Twitter. So I decided to create an automated Twitter Campaign.

If I want to play with Python - Then I can send tweets programatically.

If I want to play with Azure... I can schedule the tweets with Azure Functions

You can see the results in Part 3

Getting Content

If you haven't seen the other parts in this series the TLDR was I asked for folks with neurodiversities (ADHD, Bipolar, Dyslexia, etc) to fill out a short-survey with the last part requesting the participant to send an anonymous message to folks that may have recently been diagnosed or are considering seeking a diagnosis.

Storage Queues

I needed a way to store the message so that they could be called.

Originally I was thinking to use a NoSQL Database like Cosmos DB. I could have added the schedule day and then have the script pull the message for the current day.

This would work but it felt like a lot to spin up a Database to store 30 documents.

My next thought was to do something lighter (like SQLite) and then I remembered a talk from PyTexas about using queues in Python. While I didn't want a script running for that long I started thinking about storing the text in a queue. This is when I remember there being Azure Storage Queues.

In short, a storage queue allows you to store millions of messages (either bytes or unicode). Its primary function is to pull data off the top of the queue and either delete the message or push it to the bottom.

This was perfect because I already had a list of messages. I just needed to add them to the queue. Then my azure function would just call from the queue processing the message.



from azure.storage.queues import QueueClient
import json

queue = QueueClient.from_connection_string(
    conn_str=<CONNECTION_STRING>, queue_name=<QUEUE_NAME>
)

for idx, message in enumerate(messages):
    queue.send_message({"index": idx, "text": message})

# to receive the message

_msg = json.loads(queue.receive_message()['content'])

Creating Images

Sending text to twitter didn't seem like the greatest option. There is a character limit and studies show tweets are more engaging when you add images. This gave me the idea of creating a base image and overlaying the text on the image.

I was able to build a simple base image using Canva. Then, I overlayed the text using Pillow. Working with text on images is somwhat complicated to get perfect so I opted to create 4 categories and a ratio for each one.



{
    "md": 0.010,
    "lg": 0.012,
    "xl": 0.014,
    "2xl": 0.020,
}

Then I used that ratio to autoscale the text until the bounding box was the desired size.



draw = ImageDraw.Draw(image)
font_size = 1
font = ImageFont.truetype("assets/Lato-BoldItalic.ttf", font_size)

while font.getbbox(text)[1] < image_size_ratio:
   font_size += 1
   font = ImageFont.truetype("assets/Lato-BoldItalic.ttf", font_size)

It wasn't perfect, but it was good_enough. Then I added that index and a hashtag onto the image, and I was ready to go.

Twitter Access

The next step was sending the tweet. Twitter has an API in which you can use to interact with the application. The most basic access is free. But in order to tweet images, I needed to request elevated access. Instead of using the API directly, I opted to use the python package Tweepy

First connect your twitter account using your CLIENT AND ACCESS KEYS AND SECRETS. Then you'll need to store the image as an object using the v1 API or (API).



import tweepy
from bytes import BytesIO #This (lets you save the image in memory and not to a file)

auth = tweepy.OAuth1UserHandler(
            consumer_key=self.consumer_key,
            consumer_secret=self.consumer_secret,
            access_token=self.access_token,
            access_token_secret=self.access_token_secret,
)

image = BytesIO()
img.save(image, format="PNG")
image.seek(0)
api = tweepy.API(auth)
media = api.media_upload(file=image, filename="my_file.png")

Then you use the v2 API (Client) to send the message embedding the media.



# with the message queued message as _msg
self.twitterv2.create_tweet(
        "text": f"{_msg['index']}: {_msg['text']} #31DaysOfNeurodivergence",
        "file": media.id
        "filename": f"{_msg['index']}{_msg['text'][:10]}",
    })

Improving the Process

There were some challenges in figuring out this process. My teammate Pamela Fox and I put together a little module called AZ Queue Tweeter that expanded on this process, allowing you to not only send tweets with images to storage queues, but also supports the full range of a personal account on twitter. Here is an example of Pamela sending a Twitter poll.



import json

qt.queue_message(
    json.dumps({
        "text": "Whats your fav Python web framework?",
        "poll_options": ["Flask", "Django", "FastAPI", "All of em!"],
        "poll_duration_minutes": 60*24}
    )
)

qt.send_next_message(message_transformer=lambda msg: json.loads(msg))

This also introduced the ability to manipulate the text prior to sending it. In my script, I use spacy to segment the messages into sentences keeping the text to 280 characters.

Running the Script

So we've walked through how to make the tweet. How do we ensure that the tweets run everyday? This is where Azure functions come to play. I was able to create a simple timer function that runs once a day for the month of October.



{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "name": "mytimer",
      "type": "timerTrigger",
      "direction": "in",
      "schedule": "0 0 12 * 10 *"
    }
  ]
}

You can use the CLI or VS Code.

Then pass a function that retrieves a message, creates the image, and sends the tweet.



def main(mytimer: func.TimerRequest) -> None:

    queue = QueueTweeter(storage_auth=sa, twitter_auth=ta)

    msg = queue.send_next_message(

        message_transformer=load_message, preview_mode=False, delete_after=False

    )

<span class="c1"># Message is in format "Index - Text"



    logging.info(f"{msg} triggered at {datetime.datetime.utcnow()}")

Was this worth the effort

When I think about the problem that I had (a lot of tweets getting sent over a large period of time) I don't think I had a lot of options that were "fully automated".

I did learn some new tools and some interesting solves (looking at you PILLOW), also my overall cost for the month was less than $0.10.

Solving the problem was actually incredibly valuable as I now intend to use the same tooling to automate some of my podcasts and other content. I can also quickly modify this to use Azure queues to send message to other channels (like Discord or Teams).

I've already converted this queue into a website using Flask + HTMX.

Check it Out

Check out the repo to see how I was able to build this campaign. Finally check out AZ Queue Tweeter to create your own campaigns!

Results of #31DaysofNeurodivergence

Jay Miller — Thu, 03 Nov 2022 07:00:00 +0000

Wrapping Up 31DaysofNeuroDivergence

A back in September I shared an idea to encourage folks considering a neurodiversity diagnosing. This included a twitter campaign for the month of October under the hashtag #31DaysofNeurodivergence.

I was able to send a message on 31 messages using an automated tweet schedule I built using Azure Storage Queues and Python. The project also inspired a Python package that would make campaigns like this easier in the future!

Results

The #31DaysofNeurodivergence tweets were seen a total of 10,186 times. They received 189 likes and were retweeted 20 times.

I don't have much of a perspective on how good/bad this is. To a marketer, this may not be anything special. And there were a lot of things that I could have spent more time focusing on (like when I tweeted, a better design or color scheme). That said I'm perfectly fine with this result and here is why.

People saw it.

Let's rephrase what this data means. There were 10,186 times that someone had the opportunity to hear someone with a Neurodivergency share their story. 189 times someone resonated with the messages of a stranger and 20 times someone thought "This information is valuable and should be shared with people in my community".

My goal for this was to share a message of encouragement and I certainly did that.

#31DaysofNeurodivergence

Jay Miller — Sat, 01 Oct 2022 18:53:48 +0000

It's October and that means it's #ADHDAwarenessMonth! As I've been mentioning I collected messages of support from folks with a Neurodivergency to give you a message of support if you're newly diagnosed or have been considering seeking a diagnosis.

These messages were collected anonymously. I'll be sending one tweet a day with the following image card and the hashtag #31daysofNeurodivergence.

Whether it's ADHD, ASD, Dislexia, Bipolar, or PTSD, or any other Neurodivergence, I hope these messages will be a voice of encouragement for you this month to speak with someone.

About the folks who participated in this.

Folks were asked a bit about their journey with mental health. While this isn't a comprehensive study. It definitely matches that of the experiences I heard from others.

Many of the folks in that filled this out belonged to one or more underrepresented group in their career.

These folks were mostly also diagnosed later in life.

The majority of folks also didn't speak openly about mental health and a large group of folks were often discouraged from seeking professional help by their culture.

If you want to want to receive these messages, be sure to follow me on Twitter.

Lastly, a majority of these folks were encouraged after hearing about others with neurodivergencies. That is why we're sharing this information in hopes that we can encourage others.

Add Code of Conduct Extension Now Comes with Options!

Jay Miller — Fri, 19 Aug 2022 19:21:00 +0000

The first update to my VS Code extension Add Code of Conduct is now available!

In this update, we added the ability to choose between two commonly used Codes of Conduct:

Contributor Covenant (The existing Option)
Citizen Code of Conduct (The new Option)

This opens the door to adding several options.

This also begins the conversation around what other files should be added by default and how to add them. Speaking with my wonderful colleage, Sarah, I feel the right thing to do is to replicat this process for all of the files I think should be a standard in making open source accessible to everyone.

That said, this extension will only handle code of conducts but there is still much to be done, like adding more options and providing a way to add defaults.

I give a few shoutouts on the changelog so check that out.

New Project 31 Tweets of Neurodiverse Encouragement

Jay Miller — Wed, 17 Aug 2022 19:10:23 +0000

We've all seen motivational posts on social media designed to ~~get follows~~ brighten our days.

I've looked at these posts wondering do they actually help. I believe they initially did. But overtime, the desire for people to find a solution was only matched with peoples desire for influence.

My development philosophy is "Helpful or Fun, and in the best cases BOTH". When I create projects, my hope is that I can make things that are helpful to others. This definitely fits in the "both" category.

I've been public about getting diagnosed with ADHD as an adult. I, like others, viewed neurodivergence (ND) as a negative personality trait. This changed only after I had wonderful neurodivergent people show me that a diagnosis didn't have to come at the sacrifice of success.

All that to say, I wanted to improve how neurodivergent folks can provide encouragement one another.

I've created a form that ND folks can fill out. It's anonymous, it doesn't try to over-classify based on individual traits. It serves a few purposes:

It provides a little insight and understanding of where folks are coming from.
It allows people to openly share to their comfort.

The goal is to send a motivational message on Twitter (hopefully from an account not my own) each day. The goal is to share real messages of encouragement for folks who have been recently diagnosed or are considering seeking diagnosis or treatment.

Why Is This on a Developer Blog

First of all, developers are people, and mental health is one of the biggest reasons people exit tech.

I've learned that many folks (including myself) got into tech without knowing they were neurodivergent. This isn't a blanket statement or me trying to match successful tech folks to mental health diagnoses. It's an observation. As I speak about my diagnoses, people reach out to me sharing they were also recently diagnosed or have been considering getting diagnosed. Neurotypical folks

Also there will be some technical details in how this will be implemented. That said, the project won't work without contributions.

How You Can Help

If you are neurodivergent, please fill out the form at https://aka.ms/jm-ndform-2022. No personal identifiable information will be collected. There are a few question s I hope to share some collective results, but also the last question is a personal message that once reviewed (again no PII or hatred) will be sent out.

My goal is that enough people can share their story that we can get a full year of encouragement.

Again the form is at https://aka.ms/jm-ndform-2022.

Deploy Azure Static Web Apps using Python

Jay Miller — Wed, 03 Aug 2022 14:44:52 +0000

TLDR: Connect your repo to Azure Static Web Apps and use GitHub Actions to build your site via Python.

Modern Static Sites (popularly referred to as JAMstack Apps) provide a custom web experience for little to no cost.

I've been working on my own static site generator for going on 5 years now. It's developed a lot over time, but one thing is consistent. It's modern python!

Many folks have used tools like github pages, Netlify or Vercel, since joining Microsoft, I wanted to learn more about how to make both static and dynamic sites using Python and Azure.

I first learned about Static Web Apps (SWA) in my first week when the team was celebrating one year of the product. By design, the plan was to serve web applications by mixing HTML and optionally dynamic content via Azure functions.

Python has fallen out of favor for building static web apps, but it is still possible to deploy a static web app. Sadly, you do have to do a little more than if you were going to use Javascript.

Setup Azure Static Web Apps (The VS Code way)

You can setup Azure Static Web Apps with VS Code using the Azure Static Web Apps Getting Started. There provides a few options for web frameworks and the closest for you will be the vanilla JS route. We'll skip the first couple steps since we're not using the demo repo (or Javascript). I don't want to re-hash the tutorial; here is a quick recap:

Open your project's repo in VS Code
Install the Azure Extension and log in to your Azure account
Select the Azure Side bar menu in the RESOURCES tab
Select the create icon (plus sign)
In the prompt, choose Create Static Web App
Run through the wizard
1. connecting your GitHub account
2. creating a name for your project
3. choose your preffered region

The next step is where things begin to differ.

IMPORTANT Selecting CUSTOM as the Deployment Method.
Set your app location to "/" path of your project or wherever your Python code lives.
Set the output path to the location where your code will create HTML.

Getting your site up and running

Sadly, Azure Static Web Apps don't support running python based build commands easily, but according to the docs, we just need ONE HTML file in a folder to build a site. we can give it that file in a few different ways. Where you build your HTML will be up to you.

Build in the image

Azure uses a system called Oryx. You don't need to know too much about how it works but it looks for specific files and chooses build specs based on them. If you have a requirements.txt, Oryx will know to use Python. ¹

Using Oryx comes with a few compromises. Oryx uses a Python 3.8 build by default. You can update up to Python 3.9.7 by adding PYTHON_VERSION: "3.9.7" to the yaml file. Also, you will need to add your build steps to the PRE_BUILD_COMMAND². You can create shell script or add the steps seperated with &&.

- name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_MY_PROJECT }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "/" # App source code path
          output_location: "OUTPUT_PATH" # Built app content directory - optional
        env:
          PYTHON_VERSION: 3.9.7
          PRE_BUILD_COMMAND: 'pip --upgrade pip && pip install -r requirements.txt && python routes.py' # This can be a shell script as well

Build Before the Image

You can separate your build steps from the Azure image deployed, building it in Github Actions.

To build your environment in GH Actions you'll need to add a block to your yaml file BEFORE the Azure Build and Deploy section. You'll need to include the setup-python action and specify the python version you would like to use. Use the major version of your python version so 3.10 and NOT 3.10.5. For more information and options on this you can check out the Setup-Python GH actions repo.

Next you'll need to add the run steps. Give this section a new name and enter the commands to run.

    run: az_build.sh # My run script is routes.py

For more than one command, use |- at the beginning which each command being on its own line.

I don't think there is much of a difference performance-wise. You're code is doing the same thing, just in a different place. Use this method to bypass Python version limits of Oryx (or separate concerns in your build).

Build Before Pushing to Github

There is another way. Build your site locally and push its contents to GithHub.

You will need to manually run your build steps. The biggest advantages is complete control and you don't have to modify your yaml file.

The advantage of doing this is if you do granular changes, you don't have to rebuild the entire site.

Viewing your Site

After you push your code to GitHub. You'll be able to see your site live. If you aren't sure of the URL, check the GitHub Actions Build and it will tell you in the build steps.

You can also refresh the Azure Section of VS Code and it should be visible.

Once it's running you can add a custom domain in the Azure Portal.

Sadly I don't think this works in Azure Static Web Apps using Pipenv or Poetry. ↩
You can add also add those steps to the POST_BUILD_COMMAND. ↩

Make Inlay Type Hints in Python Appear/Disappear

Jay Miller — Wed, 27 Jul 2022 21:46:57 +0000

A Type Hinting Tip for Those Not Completely Onboard

TLDR: You can set Inlay Hints: Enabled to On/OffUnless pressed in the settings to show/hide inlay type hints in Python code.

In July the VS Code Python Team released an update for VS Code that announced inlay Type Hint Support.

Adding type hint inferences next to your code is very nice. The more I started playing with it, I noticed that sometimes the type hints didn't feel helpful and made my code look cluttered.

Don't get me wrong, I like type hints. They are a massive help with troubleshooting and documentation. They are even used in dataclasses, one of my favorite standard library tools. But as I tell my child, there is a time and place for everything. When it comes to type hints, All the Time!, is not the answer.

Let's take the following example.

Let's say we have some dictionary objects that are brought into our code from multiple systems. Sometimes the employee_id will be a numerical id and other times it will be a unique string.


jay = {
    "name": "Jay",
    "employee_id": "abcd1234" # some records could be integers depending on the schema
}

If we don't define types, PyLance will assume that the contact jay is the type dict[str, str] because all the values in the dictionary are str. What happens if we have a different record like:

kevin = {
    "name": "Kevin",
    "employee_id": 12345678
}

The variable of kevin would be typed dict[str, Any] because the type of the employee_id differs from the name.

If we build a function that gets the employee id of multiple entries and sorts by employee_id, We'll get a TypeError.

def get_employee_id(contact):
    """retrieve employee id from contact"""
    return contact["employee_id"]

sorted_employees = sorted([jay, kevin], key=get_employee_id)
>>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>

TypeError: '<' not supported between instances of 'int' and 'str'

The solution is to return the contact variable as a str. Type Hints would have told us that the contact would have been Any type (like kevin). And this would have been a hint we need to make types consistent.

We could even create a custom named type as Łukasz Langa mentions in his PyCon US 2022 Keynote. This would provide helpful hints as we're writing the code.

The July 2022 Update

The aforementioned VS Code Python update made it so that types could be inlayed next to your code. This makes adding type hints much simpler because hints (which are not added to your code) are valid Python code and can be added by the author.

In this case the hints would have been helpful but I don't want them always be present. There is an existing feature that may be new to Python developers using VS Code that will make showing your type hints only when you want to see them.

Turning On Inlay Hints

For this to work you must first turn on Inlay Hints for python. Make sure the Python Extension is installed in VS Code and search "Python Inlay Hints" in settings.

The first value to turn on is Python › Analysis › Inlay Hints: Function Return Types. This gives typing for what a function or method is returning.

The second is Python › Analysis › Inlay Hints: Variable Types. This inlays hints on variables that are written (like the ones above).

Customizing How Inlay Hints Present in Your Editor

Next in settings just search for "Inlay". You should find Editor › Inlay Hints: Enabled. The value is set to on by default, but it has a few options, including OnUnlessPressed and OffUnlessPressed.

If you change the value to OffUnlessPressed, you will no longer see inlayed hints until you enter Ctrl + Alt (⌃ + ⌥ on MacOS). When you need a hint, press the keys and the type hints will reappear. Release the keys and they disappear again.

You can also set OnUnlessPressed. This does the opposite, only showing the code that exists in the file. This entry in settings is also next to other Inlay hint stylings that may help you differentiate your code from hints.

Announcing DiversityOrgs.Tech (Beta 2)

Jay Miller — Wed, 29 Jun 2022 17:08:58 +0000

Oringally Posted on kjaymiller.com

Two years ago, I started working on Diversityorgs.Tech as a project to learn new tools and create a resource for underrepresented folks in tech everywhere. Today I'm happy to announce some amazing new features thanks to the entire project being rebuilt using Django 4.

Let’s start with exploring some of the new features.

Map Based Searching:

It’s good to know what’s in your area. Not only are we indexing city-level locations of these orgs, but you can now find them on a map. Pages with organizations in different locations will present a map so you can view the results in that area. This is powered by Azure Maps for geocoding, and map display.

Focused Searching and Filtering

We’ve all had the painful experience of looking for similar results and the service you’re using forces you to reapply all the previous settings manually. For Example, you are looking at PyLadies Atlanta and you want more groups focused on women. You select the women tag and the site will first return orgs in Atlanta with the same tag. On that page there is an option to expand to all orgs with that tag or that location.

Better Organization Management

One of the biggest barriers to further development previously was that it was very impractical to add/update content. I’ve added two new features to make organization management easier.

First, organizers can create an account and request moderator rights to an org. Once reviewed and approved, they can make updates to their org’s location, links, logo and more.

Second, anyone can suggest changes to an organization. These changes will be reviewed, and updates can be made by organizers or administrators

Learn More Than Just a Name

Joining an organization as a member is one of the first steps in getting more involved in the developer community. That said many of those foundational experiences can impact future decisions. This site highlights important things like having a publicly available code of conduct.

Speaking of good intentions, we’ve also added the ability to report any organization for removal by ANYONE. Each report is taken seriously and investigated.

The API

The REST API is generally available in limited scope with more endpoints available with an API Key. The hope is organizations and municipalities can use the data collected.

Organizers can even manage the organizations they are responsible for providing easy updates and automation.

Information Freedom and Data Privacy

This site aims to ensure that folks find helpful information to determine if an organization is right for them. All information for each organization is available for free with no need to create an account unless you are an organizer. The site doesn’t collect or sell personal identifying data nor share information with organizations.

Simply put, your data is just that, YOURS.

I’ll be working with my team at Microsoft to highlight some of the specifics of deploying and maintaining this app in the Azure ecosystem. You'll be able to see those posts in the future on the Microsoft Azure Dev.To Channel.

This was a big undertaking that took much of the last few months to get to this point. There are likely bugs and some of the interfaces will undergo some changes. If you spot an issue, please let me know by submitting an issue over on the GitHub Repo. Lastly, this project is open-source project and accepting contributions from folks at all skill-levels.

I hope that you'll enjoy using Diversityorgs.Tech and that you point underrepresented folks to this as a resource in their tech career at all stages.