Geeks With Blogs

Joe Mayo

While adding a new feature to the LINQ to Twitter Search API, I made significant changes to the Search entity that will break existing code. The new feature is support for Tweet Entities, a recent addition to Twitter’s Search API.  I’ll cover Tweet Entities support after explaining what has changed.

Motivation for Change

When Twitter implemented Tweet Entities for their Search API, they only supported JSON format.  LINQ to Twitter used ATOM for Search API queries; so, I didn’t have a choice on data formats.  What I found during analysis was that the fields of the JSON format differed significantly from the ATOM format.  More specifically, the the ATOM format contains more metadata than what’s necessary in JSON.  Since I had to make the switch anyway, I decided to refactor the naming so it matched the JSON format more closely – the benefit being that if you had to look at the Twitter API docs, the match between LINQ to Twitter Search entity property names and the Twitter API docs would be closer.  That said, in most cases, the property names are very similar or haven’t changed at all.  Overall, the changes, described below, will make the API more approachable and discoverable.

List of Changes

  • Search no longer derives from AtomFeed. Instead, all return properties are members of Search. (AtomFeed, AtomEntry, and AtomAuthor types still reside in the library, but they aren’t associated with Search anymore.)
  • Returned tweets now reside in the Results property of the Search entity (previously Entries property of AtomFeed), which is a list of SearchEntry.
  • SearchEntry has a normal ID, so you no longer need to parse it out like you did with the ID property of AtomEntry.
  • The ResultType property is now in a MetaData property, along with number of recent retweets.
  • SearchEntry has a CreatedAt property that is a DateTimeOffset of the time returned from Twitter as opposed to being translated in your local time.
  • Most existing properties correspond to existing AtomFeed properties, but many names have changed. (i.e. instead of using the Content property, you would now use the Text property)
  • Properties no longer supported (don’t appear in the JSON format) include: Alternate, Self, Title, Updated, Language (output), and TwitterWarning.
  • The RawResult property of TwitterContext will now contain a JSON formatted response (no longer an ATOM formatted response).

The impact of this change only affects how your application processes returned results from Twitter searches. All input parameters are still the same.  These changes facilitate support for Tweet Entities, described next.

The Tweet Entities Enhancement

Tweet Entities are metadata and other bits of information that Twitter extracts from a tweet, saving you the time of parsing the information yourself.  Current entities include hashtags, media, urls, and users.  Each entity has Start and End position properties in the tweet where it was extracted in addition to other entity-specific details.  The following codes shows how to find entities, showing counts of each entity type:

        public static void SearchIncludingEntities(TwitterContext twitterCtx)
            var searchResults =
                (from search in twitterCtx.Search
                 where search.Type == SearchType.Search &&
                       search.Query == "@rschu" &&
                       search.IncludeEntities == true &&
                       search.PageSize == 100
                 select search)

            searchResults.Results.ForEach(entry =>
                    Entities entities = entry.Entities;
                        "-- Entities --\n" +
                        "User: " + entities.UserMentions.Count + "\n" +
                        "Urls: " + entities.UrlMentions.Count + "\n" +
                        "Hash: " + entities.HashTagMentions.Count + "\n" +
                        "Media: " + entities.MediaMentions.Count + "\n");

The Query searches for results from someone I like to follow and whose tweets contain a variety of entity types.  Notice that I set IncludeEntities to true to ensure Twitter populates results with entities.  Here’s the description of the Search entity so you can see some of the structure and how to find entities:

- Search (this is searchResults above and contains relevant info for the Search instance)

-- … (other properties)

-- Results (List of SearchResult, containing info on tweets and metadata)

    --- Entities (Instance of Entities, containing the different entity types)

         ---- UserMentions (List of UserMention entities)

         ---- UrlMentions (List of UrlMention entities)

         ---- HashTagMentions (List of HashTagMention entities)

         ---- MediaMentions (List of MediaMention entities)

-- … (other properties)

As shown in the code snippet and hierarchy above, the Search.Results.Entities instance contains the entities you can read and use as necessary.

If you’ve previously worked with Status entities in LINQ to Twitter, you might notice similarities.  That’s because both the LINQ to Twitter Status and Search entities types are the same.

These changes haven’t been officially released, but you’re welcome to download the LINQ to Twitter source code.


You’ve seen how I refactored the Search results and how the LINQ to Twitter Search API now supports Tweet Entities. I hope you’ll see that the changes are for the better, trading a bit of discomfort for new functionality.


Posted on Monday, January 9, 2012 1:17 PM | Back to top

Copyright © Joe Mayo | Powered by: