Creating Calendar Events from Voice Commands Without Losing Your Mind

Creating Calendar Events from Voice Commands Without Losing Your Mind

ByUS Stacked

1. Capturing clean voice input without getting unusable gibberish

Voice input feels magical until you realize that what your mic hears at 7:30am isn’t exactly usable. I’ve tried using Siri, Google Assistant, and even a brief stint with a custom shortcut piped through Whisper. Each had one consistent problem: speech-to-text isn’t built for event context. You say, “Remind me to call Jenny about payroll at 2pm,” and it turns into “call journey now peril to him.”

The only way I’ve gotten anything remotely consistent is:

  • Use a specific command structure: “New meeting with [person] at [time] about [topic]”
  • Trigger a pre-defined shortcut so the assistant doesn’t try to guess the app
  • Immediately parse and normalize using an AI model (I’ve used GPT-4 base, but Anthropic’s Claude handled partial contexts better)

The Whisper API drop from OpenAI worked surprisingly well if you pair it with a prompt that pre-frames it: “This is a human scheduling a meeting. Extract name, time, date, and purpose.” But don’t expect punctuation or tense consistency—it drifts if your speech isn’t clean. Also, background noise causes it to hallucinate participants (“Call Jenny and Mariah” becomes “…and Ryan on fire…”)

2. Structuring the parsed output to fit into a calendar payload

So now you’ve got some transcribed text sitting in a Make scenario or Zap—it says something like “meeting with Jenny about payroll at 2pm Thursday.” Cool, but Google Calendar doesn’t want that. It wants:

{
  "summary": "Payroll Discussion with Jenny",
  "start": {
    "dateTime": "2024-06-20T14:00:00",
    "timeZone": "America/New_York"
  },
  "end": {
    "dateTime": "2024-06-20T14:30:00",
    "timeZone": "America/New_York"
  }
}

Parsing that reliably is more annoying than it should be. Natural language processing doesn’t like ambiguous time references. “Friday at 3” is fine unless it’s 3pm Friday now—then it parses to last week. You either enforce ISO formatting up front (never works via voice) or clean up with a date parser like Chrono or Duckling. I’ve had the best luck sending to Make with a custom function module that uses the Luxon library to guess-and-confirm the parsed datetime vs now().

Also: Google Calendar will reject malformed events silently depending on timezone conflicts—it won’t throw a plain error but instead just not create anything. Check your scenarios’ webhook logs. It took me two days to realize this was because an AI-generated event summary included emojis, and Google didn’t like emoji in some fields. That’s not documented anywhere.

3. Getting real-time triggers from the voice platform into automation flows

The mic > AI > calendar path breaks constantly on the mic side. If you’re using an iOS shortcut, good luck debugging that. One day it fires your webhook to Zapier. Next day it launches Notes and saves nothing. This gets worse if you try to use Siri Shortcuts as a service hook—Apple throttles some actions if too many calls are made back-to-back.

Here’s what I settled on when I really needed it stable:

  • iOS Shortcut triggers a webhook (hardcoded URL) to Make
  • Shortcut forces Dictation input, transcribes immediately, hits webhook
  • Webhook accepts raw text, pipes to GPT with instruction primer
  • GPT returns JSON with normalized fields (title, time, duration)
  • Make uses router to confirm that time is at least 5 mins from now (avoids late/expired events)

One weekend this broke because I updated the iOS Shortcut without realizing it unset the spoken input field—it defaulted to prompting a button instead. No error, no feedback, just… silence. Stared at my phone saying “new event tomorrow with Sam” for three days wondering why nothing showed up. If you’re using personal shortcuts, export a copy after every change and label it by VERSION or the rollback will make you weep.

4. Automatically detecting incomplete or ambiguous voice prompts

The voice flows fall apart when people say half the thing. “Add meeting with Jeff next week” has three missing pieces: time, day, duration. If I sent that directly to Google Calendar, it’d either set it at midnight or not at all. Most of the time it doesn’t make an event at all—just fails silently.

The fix: build an intermediate check into your AI layer. I added a logic check in GPT instructing it: “If required data is missing (time, date, topic), return a special message: ‘INCOMPLETE_INPUT’ with missing_fields array.” Then I route based on that. If it comes back incomplete, I send a push notification to my phone (via Pushover) prompting the user manually: “Missing date and time for Jeff meeting. Tap to fix.”

Undocumented edge case: if your AI response comes in too slowly across Zapier’s OpenAI integration, the entire Zap times out and triggers a webhook retry—which means your voice input gets double-processed. That often results in two identical calendar events. I had a week where this happened four times before I realized GPT was taking just over 10 seconds, triggering the retry behavior. Switched to Make instead with a longer timeout option, problem solved.

5. Preventing duplicate or overlapping calendar entries from one voice command

Google Calendar doesn’t enforce uniqueness. If your event summary and time are the same, it still creates a duplicate. So if Whisper or GPT slightly varies the JSON output each time, boom—two payroll meetings at 2pm with the same person.

The trick here is caching the last known voice request ID. I generate a hash of the raw voice input + timestamp, store it in a Make datastore, and check before final creation. If it already exists within +/-1 minute, I suppress creation. You could also stash it in Airtable with a similar check.

Aha moment was seeing this in my logs: an assistant said “schedule meeting for sales recap,” but background noise caused Whisper to transcribe it twice, each with 99% the same text. The webhook fired twice, events created twice, client confused. The fix? Add a deduplication filter step comparing both event summary string similarity AND start time proximity.

6. Handling multi-person scheduling when AI guesses the wrong email

This one’s delicate. You say “meeting with Brian and Krista on Thursday” and if your AI splits those into emails by guessing from prior data, Krista becomes Kristine (the recruiter) every time. I had GPT stuffing the wrong attendee emails 30% of the time because it assumed based on department list order.

Solution: Do NOT let GPT guess addresses. Instead, pass back names only and resolve to email via a lookup table (Airtable or Notion works). I keep a list of known collaborators, project-specific participants, and map by voice name to correct calendar invite email.

Then add anti-fuzz logic—if the name in the voice prompt doesn’t match confidently (Levenshtein score under 0.8 or whatever you like), push a mobile prompt: “Who do you mean by Krista? Tap to clarify.”

This killed the false invite issue. Before that, I had three weeks of calendar invites with a graphic designer who was never supposed to be in those meetings. She never RSVPed once—just let me figure it out.

7. Checklist of working pieces to recheck when the flow stops cold

This is mostly for me next time I forget all this and try to rebuild it in a hotel lobby:

  • Shortcut has correct spoken input enabled and returns text, not Dictation screen
  • Webhook URL is hardcoded cleanly (no smart quotes if copied from Apple Notes)
  • AI receives the exact raw transcription, not a formatted message object
  • Event payload uses ISO format with explicit timezones
  • Downstream tool (Calendar API) accepts the emoji or characters used in title
  • Duplicates blocked by checking hash of raw voice text plus timestamp
  • Attendee emails come only from lookup, not AI guesses

If everything checks out and your event still isn’t showing up, hold your breath and look at the logs for webhook retries—especially if OpenAI takes its sweet time responding.