Skip to content

Commit c2d8a44

Browse files
pkandrashkoupkandrashkou
committed
Release 0.0.3
2 parents a4abcc3 + 23a4163 commit c2d8a44

File tree

9 files changed

+224
-23
lines changed

9 files changed

+224
-23
lines changed

‎README.md

Lines changed: 182 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,193 @@
55
![License](https://img.shields.io/badge/license-MIT-blue.svg)
66
![Platform](https://img.shields.io/badge/platform-MacOS-lightgrey.svg)
77

8+
Netapix is a neural network framework written on pure C. For now it supports CPU mode only. The purpose of the project is investigation of deep neural networks behavior and designing new and effective architectures of convolutional networks.
89

9-
# Requirements
10+
# Installation
11+
```
12+
make
13+
```
1014

15+
# Usage
1116

12-
# Installation
17+
### Train brand new model
18+
```
19+
./example/bin/netapix train [NPX_PATH] [NPT_PATH]
20+
```
21+
Folders `weights` and `weights/params` will be created at the `[NPX_PATH]`.
22+
- `weights` for trained weights `.npw` file.
23+
- `weights/params` for copied configuration `.npx` file.
1324

1425

15-
# Usage
26+
### Continue training existing model
27+
```
28+
./example/bin/netapix train [NPX_PATH] [NPT_PATH] [NPW_PATH]
29+
```
30+
Folders `weights` and `weights/params` will be created at the `[NPW_PATH]`.
31+
- `weights` for new trained weights `.npw` file.
32+
- `weights/params` for copied configuration `.npx` and `.npw` files that were used to continue training.
33+
34+
35+
### Test the model
36+
```
37+
./example/bin/netapix run [NPI_PATH] [NPW_PATH] [OUTPUT_PATH]
38+
```
39+
Output file `.npo` will be created under `[OUTPUT_PATH]` folder.
40+
`[OUTPUT_PATH]` is optional. Default value at executable filepath – `./example/bin/output/`.
1641

1742

1843
# Documentation
44+
45+
## Formats
46+
47+
Netapix works with custom formats. There are number of tools for conversion available [here](https://github.com/touchlane/NetapixTools)
48+
49+
### .npx
50+
Special format to define network structure and learning policy. See example of simple convolutional network for MNIST dataset below:
51+
52+
```
53+
[config]
54+
batch=32
55+
threads=4
56+
channels=1
57+
width=28
58+
height=28
59+
init=xavier
60+
validation=10
61+
backup=5000
62+
learning=gradient
63+
regularization=L2
64+
accuracy=0.00001
65+
eta=0.01
66+
momentum=0
67+
lambda=0
68+
alpha=0.99
69+
beta=1.01
70+
gamma=1.01
71+
72+
[convolutional]
73+
width=14
74+
height=14
75+
channels=100
76+
stride=1
77+
padding=0
78+
activation=relu
79+
80+
[convolutional]
81+
width=14
82+
height=14
83+
channels=10
84+
stride=1
85+
padding=0
86+
activation=relu
87+
88+
[convolutional]
89+
width=2
90+
height=2
91+
channels=10
92+
stride=1
93+
padding=0
94+
activation=relu
95+
96+
[loss]
97+
input=10
98+
activation=msqe
99+
```
100+
101+
### .npw
102+
103+
Particular format for binary files with weights. Every .npw file follows the same structure:
104+
```
105+
[number of layers][4 bytes]
106+
107+
[layer config] [4 bytes * 15]
108+
[weights] [4 bytes * n]
109+
[biases] [4 bytes * m]
110+
111+
[layer config] [4 bytes * 15]
112+
[weights] [4 bytes * n_1]
113+
[biases] [4 bytes * m_1]
114+
.
115+
.
116+
.
117+
[layer config] [4 bytes * 15]
118+
[weights] [4 bytes * n_k]
119+
[biases] [4 bytes * m_l]
120+
```
121+
122+
### .npt
123+
124+
Distinct file format for train binary files of the [Netapix](https://github.com/touchlane/Netapix/) framework. It consists of two consequtive one-dimensional arrays with float32.
125+
126+
### .npi
127+
128+
Custom format for input binary files of the [Netapix](https://github.com/touchlane/Netapix/) framework. Contains two consequtive one-dimensional arrays with numbers of float32 format.
129+
130+
### .npo
131+
132+
Peculiar format for output binary files of the [Netapix](https://github.com/touchlane/Netapix/) framework. This format has one-dimensional array of 4-byte float inside.
133+
134+
## Config
135+
136+
| Key | Comment |
137+
| ------------- | ------------- |
138+
|**threads** | number of availbale CPU threads |
139+
|**batch** | total number of training examples present in a single batch |
140+
|**channels** | the depth of the input tenzor (for networks having the first layer as convolutional) |
141+
|**width** | the width of the input tenzor (for networks having the first layer as convolutional) |
142+
|**height** | the height of the input tenzor (for networks having the first layer as convolutional) |
143+
|**init** | the weights initialization type |
144+
|**validation** | indicates the part of the training set reserved for the cross validation |
145+
|**backup** | weights save rate |
146+
|**learning** | supported optimiziers (**gradient**) |
147+
|**regularization** |regularization type (**L1** or **L2**)|
148+
|**accuracy** | the target occuracy |
149+
|**eta** | start learning rate |
150+
|**momentum** | momentum coefficient|
151+
|**lambda** | regularization coefficient |
152+
|**alpha** | decrease learning rate coefficient|
153+
|**beta** | increase learning rate coefficient |
154+
|**gamma** | delta error correction coefficient |
155+
156+
## Layers
157+
158+
### Connected
159+
| Key | Comment |
160+
| ------------- | ------------- |
161+
| input | the output's size of the previous layer |
162+
| activation | the type of the activation function |
163+
164+
### Convolutional
165+
| Key | Comment |
166+
| ------------- | ------------- |
167+
| width | filter width |
168+
| height | filter height |
169+
| channels | number of filters |
170+
| stride | controls how the filter convolves around the input volume |
171+
| padding | positive integer to define the central kernel element |
172+
| activation | the type of the activation function |
173+
174+
### Loss
175+
| Key | Comment |
176+
| ------------- | ------------- |
177+
| **input** | the size of the networks's output |
178+
| **activation** | the type of a loss function |
179+
180+
## Math
181+
182+
### Activation
183+
184+
| Key | Comment |
185+
| ------------- | ------------- |
186+
| **linear** | *f(x) = x* |
187+
| **relu** | *f(x) > 0 ? x : 0* |
188+
| **logistic**| the standard logistic function|
189+
| **th**| the hyperbolic tangent |
190+
| **softmax**| the normalized exponential function |
191+
192+
### Loss
193+
194+
| Key | Comment |
195+
| ------------- | ------------- |
196+
| **msqe** | mean squared error |
197+
| **entropy** | cross entropy |

‎example/config.mk

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ EXEC = $(EXEC_PATH)/netapix
1111
# Compiler, Include, Linker Defines.
1212
CC = gcc
1313
APP_INCLUDE = -I./include/ -I.$(APP_SRC_DIR)
14-
APP_CFLAGS = $(APP_INCLUDE) -w
14+
APP_CFLAGS = $(APP_INCLUDE) -Wall -O3
1515
LIBPATH = -L./lib -lnetapix
1616
LFLAGS = -o $(EXEC) $(LIBPATH)
1717

@@ -32,7 +32,7 @@ example_mkdir:
3232

3333
# Clean Up Exectuable, Objects, Library, Coverage files d
3434
example_clean:
35-
rm -rf $(EXEC) $(APP_OBJ_DIR)
35+
rm -rf $(EXEC_PATH) $(APP_OBJ_DIR)
3636
rm -rf $(APP_SRC:.c=.gcda) $(APP_SRC:.c=.gcno)
3737

3838
.PHONY: example_all example_clean

‎example/src/netapix.c

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ int train_mode(int argc, char *argv[]) {
5050
params_save_path = make_output_save_path(weights_path, DEFAULT_OUTPUT_WEIGHTS_PARAMS_DIRECTORY_NAME);
5151
} else {
5252
output_path = make_output_save_path(npx_path, DEFAULT_OUTPUT_WEIGHTS_DIRECTORY_NAME);
53-
params_save_path = output_path;
53+
params_save_path = make_output_save_path(npx_path, DEFAULT_OUTPUT_WEIGHTS_PARAMS_DIRECTORY_NAME);
5454
}
5555

5656
if (prepare_output_path(output_path, 0) || prepare_output_path(params_save_path, 1)) {
@@ -71,18 +71,21 @@ int run_mode(int argc, char *argv[]) {
7171
}
7272
char *input_path = argv[2];
7373
char *weights_path = argv[3];
74-
char *output_path = (argc > 4) ? argv[4] : NULL;
75-
76-
output_path = make_output_save_path(output_path ? output_path : "./", DEFAULT_OUTPUT_RUN_DIRECTORY_NAME);
74+
char *output_path = (argc > 4) ? strdup(argv[4]) : NULL;
75+
76+
if (!output_path) {
77+
output_path = make_output_save_path("./", DEFAULT_OUTPUT_RUN_DIRECTORY_NAME);
78+
}
79+
7780
if (prepare_output_path(output_path, 1)) {
7881
return 0;
7982
}
80-
83+
8184
char *input_name = remove_ext(last_path_component(input_path));
8285
output_path = realloc(output_path, (strlen(output_path) +
8386
strlen(input_name) +
8487
strlen(".npo") + 1) * sizeof(*output_path));
85-
sprintf(output_path, "%s%s.npo", output_path, input_name);
88+
sprintf(output_path, "%s/%s.npo", output_path, input_name);
8689

8790
run(input_path, weights_path, output_path);
8891
return 0;

‎makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ALIB = libnetapix.a
99
# Compiler, Include, Linker Defines.
1010
CC = gcc
1111
LIB_INCLUDE = -I./include/ -I./src/
12-
LIB_CFLAGS = $(LIB_INCLUDE) -w
12+
LIB_CFLAGS = $(LIB_INCLUDE) -Wall -O3
1313

1414
all: example
1515

‎src/convolutional.c

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,12 @@ void calc_сonvolutional_сorrections(convolutional_layer *layer) {
145145
int free_convolutional_layer(convolutional_layer *layer, int is_first_layer) {
146146
if (is_first_layer) {
147147
free(layer->input);
148-
free_tensor(layer->previous_gradients_tensor, layer->h, layer->z);
149-
free_tensor(layer->input_derivative_tensor, layer->h, layer->z);
148+
// if (layer->previous_gradients_tensor != NULL) {
149+
// free_tensor(layer->previous_gradients_tensor, layer->h, layer->z);
150+
// }
151+
// if (layer->input_derivative_tensor != NULL) {
152+
// free_tensor(layer->input_derivative_tensor, layer->h, layer->z);
153+
// }
150154
}
151155
free_tensor(layer->input_tensor , layer->h, layer->z);
152156
free_array_of_tensors(layer->weights, layer->h1, layer->z1, layer->z2);

‎src/math.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ float msqe(float *vector, float *target, int lenght) {
7474
tmp = target[i] - vector[i];
7575
error = error + tmp*tmp;
7676
}
77-
return error/2;
77+
return error / lenght;
7878
}
7979

8080
float cross_entropy(float *vector, float *target, int lenght) {

‎src/run.c

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,7 +255,20 @@ int read_layer_configs_from_npw(char *path, layer_config **configs, int *layers_
255255
fclose(file);
256256
return ERROR_FORMAT;
257257
}
258-
int values_count = ((*configs)[i].input_length + 1) * (*configs)[i].output_length;
258+
int values_count = 0;
259+
layer_config config = (*configs)[i];
260+
layer_type type = (*configs)[i].type;
261+
switch (type) {
262+
case CONNECTED:
263+
values_count = ((*configs)[i].input_length + 1) * (*configs)[i].output_length;
264+
break;
265+
case CONVOLUTIONAL:
266+
values_count = config.width * config.height * config.input_depth * config.channels + config.channels;
267+
break;
268+
default:
269+
printf(FATAL_ERROR_MAKE_WEIGHTS_FAIL_MSG);
270+
break;
271+
}
259272
fseek(file, values_count * sizeof(float), SEEK_CUR);
260273
}
261274
fclose(file);

‎src/train.c

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -183,9 +183,7 @@ int train(char *npx_path, char *train_path, char *weights_path, char *output_pat
183183
int i, j;
184184

185185
int epoch = 0;
186-
printf("Initial validation...\n");
187-
root->params->prev_error = cross_validation(validation, 0);
188-
printf("Epoch:%d Iteration:0 Cross Validation Error:%f\n", epoch, root->params->prev_error);
186+
root->params->prev_error = -1;
189187
if (output_path != NULL) {
190188
char *initial_weights_path = malloc((strlen(output_path) +
191189
strlen("0.npw") + 1) * sizeof(*initial_weights_path));
@@ -195,7 +193,7 @@ int train(char *npx_path, char *train_path, char *weights_path, char *output_pat
195193
free(initial_weights_path);
196194
}
197195
int iteration = 0;
198-
while (root->params->prev_error > root->npx->settings->accuracy) {
196+
while (root->params->prev_error > root->npx->settings->accuracy || root->params->prev_error == -1) {
199197
root->params->train_set_number = epoch % root->data_set->count;
200198
shuffle(root->data_set->training_set[root->params->train_set_number]);
201199
for (i = 0; i < iterations_in_seen; i++) {
@@ -226,7 +224,11 @@ int train(char *npx_path, char *train_path, char *weights_path, char *output_pat
226224
sprintf(buffer_path, "%s%d.npw", output_path, iteration);
227225
write_npw(buffer_path, root->weights, root->npx->net, root->npx->size - 1);
228226
}
229-
printf("Iteration:%d Error:%f\n", iteration, average(root->params->batch_errors, root->npx->settings->batch));
227+
float avgError = average(root->params->batch_errors, root->npx->settings->batch);
228+
if(root->params->prev_error == -1) {
229+
root->params->prev_error = avgError;
230+
}
231+
printf("Iteration:%d Error:%f\n", iteration, avgError);
230232
}
231233
float error = cross_validation(validation, root->params->train_set_number);
232234
epoch = epoch + 1;

‎tests/src/test_math.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,8 +161,8 @@ int test_math_msqe(void) {
161161
float error3 = msqe(vector3, target, length);
162162

163163
assert_equal_float(error1, 0);
164-
assert_equal_float(error2, 1.5);
165-
assert_equal_float(error3, 2);
164+
assert_equal_float(error2, 0.75);
165+
assert_equal_float(error3, 1);
166166

167167
return 0;
168168
}

0 commit comments

Comments
 (0)