Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/sql-processor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@ name: SQL Processor CI
on: pull_request
jobs:
sql-processor-ci:
# The template only contains an unimplemented stub, run CI in student repositories only.
if: github.repository != 'course-go/sql-processor'
uses: course-go/homework/.github/workflows/sql-processor.yaml@master
10 changes: 10 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ linters:
disable:
# restrictive
- exhaustruct
- exhaustruct_v5
- varnamelen
- nlreturn
- funlen
Expand All @@ -20,6 +21,8 @@ linters:
- forbidigo
- wsl_v5
- revive
# deprecated
- gomodguard # replaced by gomodguard_v2
settings:
dupl:
threshold: 300
Expand All @@ -28,6 +31,13 @@ linters:
gosec:
excludes:
- G304 # file over variable
- G703 # path traversal via taint analysis

exclusions:
rules:
- path: _test\.go
linters:
- goconst

formatters:
enable:
Expand Down
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ the [homework](https://github.com/course-go/homework) repository.

## Assignment

Throughout this homework assignment you will create SQL processor command line tool.
Throughout this homework assignment you will create an SQL processor command line tool.
The tool receives its parameters over command line arguments and listens for SQL
file changes on local file system. These SQL files then get parsed and their SQL
file changes on the local file system. These SQL files then get parsed and their SQL
statements get processed. After processing, they get exported to a bunch of exporters.

![SQL Processor diagram](assets/sql-processor.svg)

In the repository, you will find skeleton of the application. First of all, look
In the repository, you will find a skeleton of the application. First of all, look
around. Some of the things you will need are already implemented.
You are free to change any code as long as all the tests work.
However, you should be able to finish the assignment just by implementing
Expand All @@ -30,7 +30,7 @@ sql-processor [DIRECTORY DIRECTIVE]
```

where **directory directive** represents a pair of directory path and SQL dialect
separated using colon.
separated by a colon.
If there are no directives provided or some of the directives are invalid, the
application will exit with an error.

Expand All @@ -43,13 +43,13 @@ sql-processor ./sql/files:postgres /var/local/db/mysql:mysql
The application will then read all the specified directives, parse them and will
watch for newly created files in the given directories. For observing the file
system changes, the [fsnotify](https://github.com/fsnotify/fsnotify) library will
be used. When such newly created file appear, the application processes the file's
be used. When such a newly created file appears, the application processes the file's
data and exports it.

You are safe to assume, that all the files created in the observed directories are
You can safely assume that all the files created in the observed directories are
valid SQL files that may only contain single line comments. All SQL statements
in these files always start on a new line and always end with a semicolon.
SQL statement can however span multiple lines.
SQL statements can, however, span multiple lines.

For example, this is a valid SQL file with 2 SQL statements:

Expand All @@ -63,7 +63,7 @@ WHERE o.status = 'completed';
UPDATE users SET last_login = NOW() WHERE id = 123;
```

While this file is invalid because of the 2 following reasons:
While this file is invalid for the following 2 reasons:

```sql
-- Valid comment
Expand All @@ -76,22 +76,22 @@ WHERE o.status = 'completed'; -- Nothing can follow a semicolon.
UPDATE users SET last_login = NOW() WHERE id = 123; SELECT * FROM users;
```

Your application may encountered errors during its runtime like not being
Your application may encounter errors during its runtime like not being
able to read a file etc. As we don't want to kill the app because of a single file,
most of the error processing will be done using logging. The components must
use the [slog](https://pkg.go.dev/log/slog) logger to log such errors
on appropriate level.

If you are unsure about some behaviour take the tests as the source of truth.
If you are unsure about some behaviour, take the tests as the source of truth.

### Bonus

Implement graceful shutdown. Listen to `SIGINT` and `SIGTERM` signals and
when such signal occurs, wait for you components to finish processing already
when such signal occurs, wait for your components to finish processing already
opened files. You might need to implement additional `Close` methods for some of
the components so you can reasonably control them.

For listening to the signals, checkout
For listening to the signals, check out
the [os/signal](https://pkg.go.dev/os/signal) package.

You can get up to 2 points for implementing the bonus.
Expand All @@ -118,9 +118,9 @@ Some of the Go packages worth looking into include:

- [fsnotify](https://github.com/fsnotify/fsnotify) for observing the
file system changes
- [os](https://pkg.go.dev/os) and [filepath](https://pkg.go.dev/filepath) for
interaction with file system
- [os](https://pkg.go.dev/os) and [filepath](https://pkg.go.dev/path/filepath) for
interacting with the file system
- [io](https://pkg.go.dev/io) and [bufio](https://pkg.go.dev/bufio) for interacting
with input and outputs
with inputs and outputs
- [sync](https://pkg.go.dev/sync) for synchronization
- [slog](https://pkg.go.dev/log/slog) for logging
Loading