@grammer_man who the fuck is this nigga and why u comin at me like that #Hoeassnigga

Had a spare hour last Thursday and decided to write a little twitter bot. There he is above. His name is Grammer_Man and he corrects other twitter users' misspellings, using data scraped from these Wikipedia pages.

Responses have been pouring in already, some agitated, some confused, but most positive -- which was a pleasant surprise. In any event, the minimal amount of effort in coding has paid off many times over in entertainment.

You can see who's responding at the moment by searching for @grammer_man, and also by checking his list of favourites.

Here is the (somewhat slapdash) code that powers our fearless spelling Nazi:

grabber.py

This module grabs the spelling data from Wikipedia.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import pickle

import requests
from BeautifulSoup import BeautifulSoup

def grab(letter):
    '''
    Grabs spellings from wikipedia
    '''
    url = 'http://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/%s' % letter
    html = requests.get(url).content
    soup = BeautifulSoup(html)
    bullets = soup.findAll('li')
    retval = {}
    for bullet in bullets:
        if 'plainlinks' in repr(bullet):
            values = bullet.text.split('(')
            if len(values) == 2:
                retval[values[0]] = values[1][:-1] # shave off the ) at end
    return retval

def get_spellings():
    '''
    Returns a dictionary of {false: correct} spellings
    '''
    if not os.path.exists('words.pkl'):
        retval = {}
        for c in 'ABCDEFGHIJKLMNOPQRSTUVWXYZ':
            print 'Getting typos - %s' % c
            retval.update(grab(c))
        print 'Dumping...'
        f = open('words.pkl', 'w')
        pickle.dump(retval, f)
        f.close()
        return retval
    else:
        f = open('words.pkl', 'r')
        retval = pickle.load(f)
        f.close()
        return retval

if __name__ == '__main__':
    get_spellings()

bot.py

The bot. Selects misspellings at random, searches for them, responds to them, while also taking breaks between tweets and longer breaks every few hours.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import random
import time
import pickle

import twitter

from grabber import get_spellings

API = twitter.Api()

MESSAGES = u'''
Hey $USERNAME, didn't you mean $CORRECT there?
#
# All messages stored in here, one per line.
#
'''.split('\n')

def compose_message(twitter_post, mistake, correct):
    '''
    Choose a message from MESSAGES at random, substitute fields to personalise it and 
    check if it exceeds the twitter message limit. Try this 100 times before failing.
    '''
    retries = 0
    while retries < 100:
        message = MESSAGES[random.randint(0, len(MESSAGES) - 1)]
        message = message.replace('$USERNAME', '@%s' % twitter_post.user.screen_name)
        message = message.replace('$MISTAKE', '"%s"' % mistake).replace('$CORRECT', '"%s"' % correct)
        if message and len(message) < 141:
            return message
    return None

def correct_spelling(twitter_post, mistake, correct):
    '''
    Correct someone's spelling in a twitter_post
    '''
    print u'Correcting @%s for using %s...' %(twitter_post.user.screen_name, 
                                            mistake)
    message = compose_message(twitter_post, mistake, correct)
    if not message:
        print u'All messages were too long... Aborting...'
        return False
    else:
        failures = 0
        try:
            API.PostUpdate(message, in_reply_to_status_id=twitter_post.id)
        except Exception, e:
            print 'Failed to submit tweet (%s).'
            return False
        return True

def search(word):
    '''
    Search twitter for uses of a word, return one if it's been used recently.
    Otherwise return None.

    TODO: Add time awareness.
    '''
    print 'Searching for uses of %s...' % word
    results = API.GetSearch(word)
    if results:
        for result in results:
            if not check_if_done(result.id) and\
                not result.user.screen_name == 'grammer_man' and word in result.text:
                return result
    return None

def check_if_done(id):
    '''
    Checks if a tweet has already been responded to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
        if id in done:
            return True
    return False

def update_done(id):
    '''
    Updates a list of tweets that've been replied to
    '''
    if os.path.exists('done.pkl'):
        f = open('done.pkl', 'r')
        done = pickle.load(f)
        f.close()
    else:
        done = []

    done.append(id)

    f = open('done.pkl', 'w')
    pickle.dump(done, f)
    f.close()

def main():
    '''
    Main program flow
    '''
    words = get_spellings()
    counter = 0 
    while True:
        word = random.choice(words.keys())
        post = search(word)
        if counter > 100:
            rand_time = random.randint(120*60, 240*60)
            print 'Done %s tweets, sleeping for %s minutes' % (counter, rand_time/60)
            time.sleep(rand_time)
            counter = 0
        # TODO: PROPERLY PRUNE THE MISTAKES/CORRECTIONS FROM WIKIPEDIA AND REMOVE THIS:
        if not u',' in word + words[word] and not u';' in word + words[word]:
            if post:
                result = correct_spelling(post, word, words[word])
                if result:
                    counter += 1
                    print '#%s Done' % counter
                    update_done(post.id)
                    time.sleep(random.randint(300,500))


if __name__ == '__main__':
    main()

Grammer_Man uses the following libraries:

• python-twitter (Be warned: no proxy support)
• requests
• BeautifulSoup

Comments

Poetry

Comments

Comb-me quickly; don't put me so much pomatum. What news tell me? all hairs dresser are newsmonger.

The title of this blogpost is taken from English As She Is Spoke, a 19th century Portugese-English phrase book which I've spent the last hour reading on the train. It's living up to all expectations. You can enjoy the entire book here.

Comments

Qaeda Quality Question Quickly Quickly Quiet

I rather like the "A", "Amer-kawh", "and", "for" and "great" bits.

Comments

But what does it mean?

From a collective of people including the man behind King Lud's Revenge.

Comments

Yo dawg, I heard you like linguistics...

Came across this amusing Inception / Yo Dawg meme face-off the other day on reddit. It led me back to one of my favourite linguistic peculiarities, the sentence "Buffalo buffalo buffalo buffalo buffalo buffalo buffalo." Its Wikipedia page seems to have been updated since the last time I had a look at it, as there are a few other interesting linguistic cases linked, none of which I'd come across before. I especially enjoyed the sentence "James while John had had had had had had had had had had had a better effect on the teacher". And then I stumbled across this old Chinese poem, "The Lion-eating Poet in the Snow Den":

The text, although written in Classical Chinese, can be easily comprehended by most educated readers. However, changes in pronunciation over 2,500 years resulted in a large degree of homophony in Classical Chinese, so the poem becomes completely incomprehensible when spoken out in Standard Mandarin or when written romanized in Standard Mandarin.

Pretty remarkable. Its Wikipedia page, linked above, is rather detailed and well worth a read.

Here's a video of the poem being read aloud in Standard Mandarin:

More: A List of Linguistics Example Sentences

Comments

Welcome to Reykjavík

An eccentric little piece from Reykjavík's Mayor. Click to enlarge.

Thanks to Hugh for bringing this to my attention.

Comments

Writing without reading

A curious case of a professional writer who awoke one morning to find his capacity to read crippled by a stroke. Animation and narration from Lev Yilmaz. You can watch the video here. For some reason the embedding seems to be a bit mucked-up.

Comments

Do you think it will always be like this?

Thanks to Paddy for bringing this to my attention. (You should really check his blog out too, it's excellent.)

Please Say Something is a 10 minute short concerning a troubled relationship between a Cat and Mouse set in the distant Future. The final film was completed in January 2009 and contains 23 episodes of exactly 25 seconds each.

The film won the Golden Bear for best short at the 2009 Berlinale, the Cartoon D’or and several other awards. In 2010 it was given a distinction of cultural significance by the German ratings agency FBW (Prädikat Besonderes Wertvoll).

Comments

True terror is to wake up one morning and discover that your high school class is running the country

Photo: Reuters

A select few quotations from a BBC article on Hitler's bizarre popularity in India:

Latest reports say Bollywood is now planning to cash in. A film - Dear Friend Hitler - is due to be released by the end of the year, focusing on the dictator's relationship with his mistress Eva Braun.

...

It's hard to narrow down what makes the dictator popular in India, but some young people say they are attracted by his "discipline and patriotism".

Most of them are, however, quick to add that they do not approve of his racial prejudices and the Holocaust in which millions of Jews were killed.

...

Nearly all the booksellers and publishers contacted in India say it is mainly young people who read Mein Kampf. It's not just the autobiography - books on the Nazi leader, T-shirts, bags, bandanas and key-rings are also in demand. A shop in Pune, called Teens, says it sells nearly 100 T-shirts a month with Hitler's image on them.

Dimple Kumari, a research associate in Pune, has not read Mein Kampf but she would wear the Hitler T-shirt out of admiration for him. She calls him "a legend" and tries to put her admiration for him in perspective: "The killing of Jews was not good, but everybody has a positive and negative side."

I have to say, I find this peculiar naivety fascinating. I also can't imagine what it must be like for a Western traveller to be walking down a street in, say, Bangalore, spotting a few people coming towards him clad in Hitler Apparel. Indeed, staying with Bangalore, since it's such a huge IT hub... Should we expect to see originally well-meaning and innocuous (to Indians, that is) photographs of young IT workers on their IBM or Microsoft campus, posing happily with their corporate swipe-cards dangling from from their neck, the strap perfectly framing a portrait of their "Dear Friend Hitler"? Indeed, do such places, renowned for their lack of dress-code in the West, already have a strict dress-code in places like India, in order to prevent such embarrassments? I wonder.

And, before I go, here's another great article from Der Spiegel on the same phenomenon, only this time in Pakistan. Yep, they're at it too. Who knows - perhaps this new-found love for the 20th century's most hated, genocidal dictator will only serve to foster a new friendship of shared values between India and Pakistan, leading to a stable peace! Surely no harm could come of future generations of two of the world's most antagonistic and unstable nuclear-countries worshipping a genocidal, maniacal, militaristic dictator!

Brings a whole new sense to that Vonnegut quote...

Comments