Navigate to desired directory to execute the program and test cases. This directory will be called as $PROJECT_ROOT

Initiate git:

git init

Associate current directory with remote repository:

git remote add origin https://github.com/saurabmish/Twitter-Sentiment-Analysis.git

Fetch project from remote repository:

git pull origin master

Initiate Python 3 virtual environment:

python3 -m venv twitter-sentiment

Activate newly created virtual environment:

source twitter-sentiment/bin/activate

Update pip:

pip install --upgrade pip

Install required packages in the virtual environment:

pip install --requirement requirements.txt

Set environment variables (convenient way of setting PYTHONPATH) :

source .env

Download and install Elasticsearch and Kibana

Start services for elasticsearch and kibana:

Linux:

sudo -i service elasticsearch start
sudo -i service kibana start

macOS:

brew services start elasticsearch-full
brew services start kibana-full

Check Elasticsearch status in the terminal:

curl -X GET "localhost:9200/?pretty"

Expected Output:

{
  "name" : "thinkpad-t480",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "BZSMYSWPRp26y_pniprZVw",
  "version" : {
    "number" : "7.10.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "51e9d6f22758d0374a0f3f5c6e8f3a7997850f96",
    "build_date" : "2020-11-09T21:30:33.964949Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Check Kibana status in the web browser:

http://localhost:5601/app/dev_tools#/console?load_from=https:/www.elastic.co/guide/en/elasticsearch/reference/7.10/snippets/19.console

To run test cases, execute the below command in $PROJECT_ROOT:

pytest -v

To check coverage (source code and test cases), execute the below command in $PROJECT_ROOT:

pytest --cov -v

Run flask microservices:

flask run

On the web browser, go to:

http://127.0.0.1:5500/

Setup monitoring:

• Go to the Elasticsearch dashboard

• Scroll down to Management in the dock navigation menu

• Click on Stack Management

• Under the Kibana section, click on "Index Patterns"

• The index attribute associated with the elasticsearch.index object will show up; select it and verify the fields

• Click on the dock navigation menu and go to Discover under the Kibana section

  • Here the data streamed by app/streamer.py will be displayed.

  • Figure out data which will be visualized on the x and y axis.

• Now click on the dock navigation menu and go to Dashboard under the Kibana section

  • Create the desired visualization