Log Analysis

The project is a internal reporting tool that use information from database to discover what kind of articles the site's readers like.

The database includes three tables:

• The authors table includes information about the authors of articles.
• The articles table includes the articles.
• The log table includes one entry for each time a user has accessed the site indicates whether the request succeeded or failed.

The reports View used in this project:

• reports is used to help to answer the first two questions so it contian articles and it's views , query of this view:

  create view reports as 
  (
  select articles.slug, count(log.path) as views 
  from articles left join log 
  on slug = substring(path from 10) 
  group by slug 
  );
  

The project consist of Four functions:

• The execute_query() Function Used To Execute Different Queries.
• The print_top_articles() Function Used To Prints out the top 3 articles of all time.
• The print_top_authors() Function Used To Prints a list of authors ranked by article views.
• The print_errors_over_one() Function Used To Prints out the days where more than 1% errors.

How it works:

1. you should install python3 from here.
2. You should have the following :
3. vagrant installed on your computer download it from here.
4. Virual Box installed on you computer downlad it from here Hint your virtualbox must be old verison 5.1. .. Newer versions do not work with the current release of Vagrant.
5. Next, download the data here. The file newsdata.sql should be inside the vagrant directory. here
6. To execute sql queries of newsdata.sql , cd into the vagrant directory and use the command psql -d news -f newsdata.sql.
7. Next cd to the project location and run python newsdb.py.