Skip to content

Commit

Permalink
Benchmarked performance, Docker updated, readme updated, some comments.
Browse files Browse the repository at this point in the history
  • Loading branch information
VikingPingvin committed Jan 9, 2021
1 parent bcf2b65 commit 88676e1
Show file tree
Hide file tree
Showing 4 changed files with 59 additions and 4 deletions.
5 changes: 5 additions & 0 deletions Docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ services:
image: locker:test
ports:
- "27001:27001"
environment:
- LOCKER_SERVER_IP="127.0.0.1"
- LOCKER_SERVER_PORT="27001"
- LOCKER_SERVER_LOG="./locker-server.log"
- LOCKER_SERVER_ARTIFACTS_ROOT="/out"
volumes:
- ./out:/out
- logvolume01:/var/log
Expand Down
1 change: 1 addition & 0 deletions locker/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ func sendArtifactToServer(artifact *ArtifactData, agentConfig *AgentConfig, wg *
Msg("Agent connected to Locker Server.")

// Send Metadata message
//TODO: Parse return err (eg.: file not found, don't continue)
parseAndSendMetaData(connection, artifact)

// Send Payload message(s)
Expand Down
2 changes: 2 additions & 0 deletions locker/agent_functions.go
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,8 @@ func parseInputArguments() []*ArtifactData {
log.Debug().Msgf("Relative path of input: %s", path)
inputPath = filepath.Join(cwd, path)

} else {
inputPath = path
}

dataArray[i] = &ArtifactData{
Expand Down
55 changes: 51 additions & 4 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,57 @@ Uses TCP/IP connection for sending file metadata and payload using protobuf betw
[![Go Report Card](https://goreportcard.com/badge/github.com/VikingPingvin/go-locker)](https://goreportcard.com/report/github.com/VikingPingvin/go-locker)
[![Go Version](https://img.shields.io/github/go-mod/go-version/VikingPingvin/go-locker)](https://img.shields.io/github/go-mod/go-version/VikingPingvin/go-locker)

## Benchmarks
**Avg.Time** is the duration from the start of the TCP connection, until the end of it, and as such, includes the complete file chunking, sending, reassemblyng server-side,
hashing and sending an ACK message to the agent, followed by the closing of the TCP connection.

<details>
<summary>For reference the HDD used for localhost benchmarking </summary>
<br>
Western Digital Caviar Blue 3.5 1TB 7200rpm 64MB SATA3
<hr>
</details>

#### Localhost Agent -> Localhost Server

|File size|Avg. Time| Write Speed |
|---------|---------|-------------|
|3.59 Gb | 162s | ~22 MB/s |
|615 MB | 23.6s | ~26 MB/s |
|393 MB | 14.5s | ~27 MB/s |
|196 MB | 8s | ~24.5 MB/s |
|47.5 MB | 1.82s | ~26 MB/s |

#### Localhost Agent -> Localhost Docker Server
With **Bind** mounted output directory.
|File size|Avg. Time| Write Speed |
|---------|---------|-------------|
|196 MB | 15s | ~13 MB/s |
|47.5 MB | 3.7s | ~12.8 MB/s |

#### Localhost Agent -> Cloud VM (Linode Nanode 1GB) native Server App

|File size|Avg. Time| Write Speed |
|---------|---------|-------------|
|196 MB | 77s | ~2.5 MB/s |
|47.5 MB | 22s | ~2.1 MB/s |

### Localhost Agent -> Cloud VM using Docker Server
Not tested

## Usage example


### Start server
> ./locker server
> .\locker.exe server -c="./cfg-server.yml"
To start Locker in server mode use the `server` command.
The server app handles initial configuration and database connection and waits for incoming connections.
### Start agent
> ./locker agent --file="path-to-file" --namespace="namespace/project/job-id
To start Locker in Agent mode, use the `agent` command.
You can supply multiple files for the `--file` flag separated with **comma**.
The agent parses the file given under the `--file` flag and collects metadata, such as sha256 hash.
It sends the metadata to the server, followed by payload messages containing the raw binary file contents.

Expand All @@ -42,26 +80,35 @@ agentconfig:
serverconfig:
server_ip: "127.0.0.1"
server_port: "27001"
artifacts_root_dir: "./out" #not yet implemented
artifacts_root_dir: "./out"
log_path: "./locker_server_log.txt" #not yet implemented
```
If no config file is supplies via `-c` flag, the above values are used as default.
### With ENV variables
If no ENV vars are supplied, the following are used as defaults, if not set otherwise:
```yml
# AGENT
LOCKER_SERVER_IP = "127.0.0.1"
LOCKER_SERVER_IP = "127.0.0.1" # The server to connect to
LOCKER_SERVER_PORT = "27001"
LOCKER_AGENT_CONCURRENT = true
LOCKER_AGENT_LOG = "./locker-agent.log"
```
```yml
# AGENT
# SERVER
LOCKER_SERVER_IP = "127.0.0.1"
LOCKER_SERVER_PORT = "27001"
LOCKER_SERVER_LOG = "./locker-server.log"
LOCKER_SERVER_ARTIFACTS_ROOT = "."
```

## Using with Docker
An example `docker-compose.yml` is supplied here:
[Docker Compose Readme](Docker/readme.md)
Disclaimer: The example docker related files are not final and WILL change over the time, only there to test functionality.

## Building
To build from source:
> go build .

To cross-compile, follow the Go compiler instruction for `GOOS` and `GOARCH` enviroment variables.

0 comments on commit 88676e1

Please sign in to comment.