Versions Compared

Old Version 1

changes.mady.by.user Rakesh Chaudhary

Saved on

compared with

New Version 2

changes.mady.by.user Rakesh Chaudhary

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
maxLevel3
absoluteUrltrue

...

  1. The user triggers the application by providing an input sentence.
  2. The UI (HTML + JavaScript) calls a REST API, which points to an Ultra Task.
  3. The Ultra Task sends the input sentence to the Twitter Sentiment Analysis Pipeline.
  4. The Twitter Sentiment Analysis Pipeline uses the Twitter Sentiment Analysis model built using SnapLogic Data Science to determine the polarity of the input sentence and send it back to the UI.
  5. The Twitter Sentiment Analysis model is built using two Pipelines:
    1. The Prepare Twitter Data Pipeline uses the sentiment140 dataset as a source of input sentences and applies Bag of Words techniques to prepare the dataset, which can be used to train the model.
    2. The Build Model Pipeline uses the processed dataset to create (and save in the SnapLogic File System (SLFS)) a model using a range of machine learning algorithms. For details on these algorithms, see the Build Model Pipeline section below.

...

PipelineDescription


Data Preparation

  1. This Pipeline reads a ZIP file containing the sentiment140 sentences, parses the data, and marks all sentences whose sentiment value is 0 as negative. All others it marks positive.
  2. The Pipeline then tokenizes all the input sentences, retaining only words, and then lists out the 200 most common words in the dataset in the sentences in which they appear, exporting them as vectors. The Pipeline also writes out a copy of the list of common words into a separate file for later use in the process.
  3. Finally, the Pipeline shuffles these rows of vectors, converts them into compressed JSON, and writes them to the SnapLogic File System (SLFS).

Build Model

  1. This Pipeline reads the processed Twitter data file created by the Data Preparation Pipeline above and builds a sentiment analysis model using the AutoML Snap.
  2. The Pipeline then writes the first output, which is the model itself, as a JSON file, into the SnapLogic File System. This is the model that the third Pipeline uses to analyze the polarity of the sentences coming from the web application. 

    Moreover, the second output provides a leaderboard, which includes statistics of all models built during the AutoML process. The third input provides a visual report of the top-ten models that the AutoML Snap built.

Twitter Sentiment Analysis 

  1. This Pipeline receives the input data from the web application via the Ultra Task. It then performs a simple authentication to check whether the user who sent the request is authorized to run the Pipeline.
  2. It then extracts and tokenizes the actual text string that comprises the input and creates a vector using its words.
  3. The Pipeline now uses the model created in the Build Model Pipeline to determine the sentiment in the input sentence and the confidence level associated with the polarity identified. 
  4. The Pipeline then sends the polarity and confidence numbers back to the web application as an API response, which are then displayed in the web application UI.
Note

The Sample Request Snap is disabled since this pipeline is scheduled as Ultra Task to provide API to external application so the data will come from the API call. This Snap contains a sample request for testing purpose, you can enable the Snap and validate the pipeline to see the result.


...

  1. Download and configure the Pipelines as discussed in the previous section. If you have already done so, you can skip the following two steps.
    1. Download the Twitter Sentiment Analysis.zip file. This file is an export of a project containing the required Pipelines, tasks, and files.
    2. Import the project. In SnapLogic Manager, select the project space > Import. For more information, see How to Import and Export Projects.
  2. Go to the project that you created in Manager and click the Tasks tab. Click the task entity in the list of files to open the Update Task dialog.



  3. Select the Snaplex on which you want to deploy this Ultra Task. Click Update.
    This updates the task and redisplays the list of files associated with the project.

    Note

    The FeedMaster node is required for scheduling Pipelines as Ultra Tasks.





  4. Your Ultra Task is now created and can be used once it is enabled. To enable the task, select the check box associated with the task and click Enable on the top-right section of the screen.



  5. You now need to copy the URL to this task, so you can use it with the web application. Click the inverted triangle next to the Task's name and click Details to open the Task details pane.



  6. In the Task details page, copy the URL listed in the HTTP Endpoint field and save it on your local machine to add it later on to your HTML file. Tasks can take up to 5 minutes to start. At any time, you can click the Details option to view the Task's status. 

...

For information on how the HTML file is coded, see Creating and Deploying the Twitter Sentiment Analysis FrontendFront-end. The page also contains information on how you can modify the HTML file before you deploy it.

...