Pipeline: Who is Tweeting About Us

On this Page

Scenario

I want to see who is tweeting what about our company and count how often they are.
This is a more complex version of  Pipeline: Count of Who is Tweeting About Us. 

Requirements

Snaps Used

For this scenario, the following Snaps are used:

  • Twitter Query
  • Structure
  • Conditional
  • Copy
  • Aggregate
  • JSON Formatter
  • CSV Formatter
  • File Writer

Other Requirements

  • A Twitter account configured in SnapLogic Manager. See Accounts for more information on configuring an account.


Configuration


  1. Create a new pipeline by clicking the (+) tab.
     
  2. Add and configure a Twitter Query Snap.
    • The the Accounts tab, select your configured Twitter account.
    • For the Twitter query field, enter the query you want to use. For this use case, set the query to @SnapLogic.
       
  3. Add and configure a Structure Snap. For this use case, we want to see who is tweeting what, but don't need all of the details Twitter provides with our query.
    • Set Pass through to [None] - new map.
    • Add two rows to the Mapping table, with the following values:
      • $fromUser, move, $fromUser
      • $text, move, $text
         
  4. Add and configure a Conditional Snap. In this use case, we want to determine which tweets are coming from internal groups versus individuals.
    • Add two rows to the Conditional table, with the following values:
      • $fromUser == "SnapLogic", "Marketing", $Who
      • $fromUser == "SnapLogicDocs", "Documentation", $Who
    • Set Default Value to "Individual"

    • Set Default Value Path to $Who

  5. Add and configure a Copy Snap. This is added so that we can work with the data in two different ways.
    • Add a second output view. Name it Count for clarification.
  6. Off of output0:
    1. Add a JSON Formatter Snap.
    2. Add and configure a File Writer Snap. This will create a file with each tweet and who it was from.
      • For each output, set the File Name to the name of the file where you want the results written to, including a path if necessary.
  7. Off of Count:
    1. Add and configure an Aggregate Snap. We will use this to see how many tweets come from each group we defined in the Conditional Snap.
      • Set Aggregate function to COUNT.
      • Set Aggregate to $Who.
      • Set Result to Total.
      • Add two rows to the GROUP-BY fields table:
        • $WhoWho
        • <null>, Total
    2. Add a CSV Formatter Snap.
    3. Add and configure a File Writer Snap. This will create a CSV file with the number of tweets for each of the 3 groups listed in the Conditional Snap.
      • For each output, set the File Name to the name of the file where you want the results written to, including a path if necessary.


By reviewing the list of users in the output0 file, you can then go back and update the Conditional Snap to mark individuals as "Employees," "Investors," "Customers," if you want to find completely unique individuals.
 

Another variation of this pipeline would be to modify the Count branch to count how many time individuals or groups tweet. To do this:
 

  1. Add $user.name to the Structure Snap. This will pull in the user name in addition to the login.
  2. Remove the Conditional Snap and join the Structure Snap to the Copy Snap.
  3. Add and configure a Mapper (Data) Snap before Aggregate. We want to combine user name and login into one column for future use
    • In the Mapping table, add a row with the values:
      • $name.concat(" ", $login)$Who
  4. Optionally, add a Sort Snap after the Aggregate Snap to sort $Total in descending order to see who tweeted the most.

The resulting file from the Count branch will now tell you how many tweets per user. Because it is only two columns of data, you can easily graph the results.