Skip to content

Commit f389c88

Browse files
robcarver17robcarver17
committed
version 0.19
1 parent ba54f1b commit f389c88

17 files changed

+857
-39
lines changed

‎README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -90,5 +90,6 @@ A series of examples using pysystemtrade for my blog posts can be found [here](h
9090
GNU v3
9191
( See [LICENSE](LICENSE) )
9292

93-
Absolutely no warranty is implied with this product. Use at your own risk. I provide no guarantee that it will be profitable, or that it won't lose all your money very quickly, or delete every file on your computer (by the way: it's not *supposed* to do that. Just in case you thought it was.).
93+
Absolutely no warranty is implied with this product. Use at your own risk. I provide no guarantee that it will be profitable, or that it won't lose all your money very quickly, or delete every file on your computer (by the way: it's not *supposed* to do that. Just in case you thought it was.). All financial trading offers the possibility of loss. Leveraged trading, such as futures trading, may result in you losing all your money, and still owing more. Backtested results are no guarantee of future performance. I can take no responsibility for any losses caused by live trading using pysystemtrade. Use at your own risk. I am not registered or authorised by any financial regulator.
94+
9495

‎docs/IB.md

+257
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
This document is specifically about using pysystemtrade to connect with [*Interactive Brokers (IB)*](https://www.interactivebrokers.com/).
2+
3+
Although this document is about Interactive Brokers, you should read it carefully if you plan to use other brokers as it explains how to modify the various classes to achieve that, or perhaps if you want to use an alternative python layer to talk to the IB API, such as the excellent [ib_insync](https://github.com/erdewit/ib_insync). Currently the IB interface covers the following methods:
4+
5+
- Get spot FX price data
6+
7+
Related documents:
8+
9+
- [Storing futures and spot FX data](/docs/futures.md)
10+
- [Using pysystemtrade as a production trading environment](/docs/production.md)
11+
- [Main user guide](/docs/userguide.md)
12+
13+
*IMPORTANT: Make sure you know what you are doing. All financial trading offers the possibility of loss. Leveraged trading, such as futures trading, may result in you losing all your money, and still owing more. Backtested results are no guarantee of future performance. No warranty is offered or implied for this software. I can take no responsibility for any losses caused by live trading using pysystemtrade. Use at your own risk.*
14+
15+
Table of Contents
16+
=================
17+
18+
* [Quick start](#quick-start)
19+
* [Getting started with interactive brokers](#getting-started-with-interactive-brokers)
20+
* [Gateway / TWS](#gateway--tws)
21+
* [Python library](#python-library)
22+
* [Launching and configuring the Gateway](#launching-and-configuring-the-gateway)
23+
* [Making a connection](#making-a-connection)
24+
* [Get some data](#get-some-data)
25+
* [How to...](#how-to)
26+
* [Connections](#connections)
27+
* [Creating and closing connection objects](#creating-and-closing-connection-objects)
28+
* [Make multiple connections](#make-multiple-connections)
29+
* [Deal with errors](#deal-with-errors)
30+
* [Get spot FX prices](#get-spot-fx-prices)
31+
* [Reference](#reference)
32+
* [Classes and object references](#classes-and-object-references)
33+
* [Client objects](#client-objects)
34+
* [Generic client object](#generic-client-object)
35+
* [IB client object](#ib-client-object)
36+
* [Server objects](#server-objects)
37+
* [Generic server object](#generic-server-object)
38+
* [IB server object](#ib-server-object)
39+
* [Connection objects](#connection-objects)
40+
* [Data source objects](#data-source-objects)
41+
* [Spot FX](#spot-fx)
42+
43+
Created by [gh-md-toc](https://github.com/ekalinin/github-markdown-toc)
44+
45+
# Quick start
46+
47+
## Getting started with interactive brokers
48+
49+
You may want to read [my blog posts](https://qoppac.blogspot.com/2017/03/interactive-brokers-native-python-api.html) to understand more about what is going on if it's your first experience of IB's python API. For any issues with IB go to [this group](https://groups.io/g/twsapi). IB also have a [webinar](https://register.gotowebinar.com/register/5481173598715649281) for the API. Finally the official manual for the IB API is [here](http://interactivebrokers.github.io/tws-api/introduction.html).
50+
51+
### Gateway / TWS
52+
53+
You need to download eithier the gateway or TWS software from the IB website. I recommend using the Gateway as it is much more stable and lightweight, and does not regularly reboot itself.
54+
55+
These links may break or become outdated - use google to find the appropriate page on IB's website.
56+
57+
[For Windows](https://www.interactivebrokers.co.uk/en/index.php?f=1341)
58+
[For Linux](https://www.interactivebrokers.co.uk/en/index.php?f=16454)
59+
60+
61+
### Python library
62+
63+
You also need the python library for IB. This can be downloaded from [here](https://interactivebrokers.github.io/#). Once you have the source code you will need to install it. Here's the Linux way:
64+
65+
```
66+
cd ~/IBJts/source/pythonclient
67+
python3 setup.py install
68+
```
69+
70+
The directory required may vary, and you might need to prefix the second command with `sudo`. Windows users should do whatever they normally do to install python packages.
71+
72+
## Launching and configuring the Gateway
73+
74+
Before you run any python code you'll need to launch the Gateway software. Current versions of the Gateway do this via a desktop icon. You will need to use eithier:
75+
76+
- A demo account, such as username: fdemo, password: demouser. IB seem to be phasing out their demo accounts.
77+
- A paper trading account
78+
- A live trading account (*Make sure you know what you are doing. I can take no responsibility for any losses caused by live trading using pysystemtrade. Use at your own risk!!!*)
79+
80+
You will also need to configure the Gateway:
81+
82+
- Socket port: Should be 4001, unless you want to change this HERE
83+
- White list for trusted IP addresses: Should include 127.0.0.1. If you are going to be running the Gateway on one machine, and accessing it via another, then you need to add the IP address of your other machines here.
84+
- If you are going to be trading, then 'Read only API' should be turned off
85+
- You may also need to change precautions and preset options
86+
87+
## Making a connection
88+
89+
```
90+
from sysbrokers.connections import connectionIB, ibConnectionConfig
91+
config = ibConnectionConfig(ipaddress = "127.0.0.1", portid=4001, client = 1)
92+
conn = connectionIB(config)
93+
conn
94+
95+
Out[13]: ibconnection(ipaddress='127.0.0.1', portid=4001, client=1)
96+
97+
```
98+
99+
## Get some data
100+
101+
FX data: `conn.broker_get_fx_data("GBP")`
102+
103+
# How to...
104+
105+
## Connections
106+
107+
### Creating and closing connection objects
108+
109+
```
110+
from sysbrokers.IB.ibConnection import connectionIB, ibConnectionConfig
111+
config = ibConnectionConfig(ipaddress = "127.0.0.1", portid=4001, client = 1)
112+
```
113+
114+
Portid should match that in the Gateway configuration. Client ids must not be duplicated by an already connected python process (even if it's hung...). The IP address shown means 'this machine'; only change this if you are planning to run the Gateway on a different network machine.
115+
116+
```
117+
conn = connectionIB(config)
118+
```
119+
120+
Connection objects immediately try and connect to IB, and kick off a server thread. So don't create them until you are ready to do this. Once you have a connection object that exists in a particular Python process, do not try and create a new one. I've also had problems closing connections and then trying to create a new connection object. Generally it is safer to stick to the pattern of creating a single connection object in each process, attempting to close it out of politeness with `conn.disconnnect()`, and then terminating the process.
121+
122+
### Make multiple connections
123+
124+
It's possible to have multiple connections to the IB Gateway, each from it's own process, but each connection must have a unique clientid (NOTE: NEED TO ADD FUNCTIONALITY TO ENFORCE THIS).
125+
126+
## Deal with errors
127+
128+
The IB and broker objects don't raise exceptions caused by IB reported errors, or any issues, but they do log them. Generally the pattern is for a call to a client method to return an empty object, which the calling function can decide how to deal with.
129+
130+
NOTE: NEED TO ADD FUNCTIONALITY AROUND A HEIRARCHY OF ERRORS (MESSAGE, WARNING, ERROR...)
131+
132+
## Get spot FX prices
133+
134+
We treat IB as another data source, which means it has to conform to the data object API (see [storing futures and spot FX data](/docs/futures.md)) for spot FX.
135+
136+
```
137+
from sysbrokers.IB.ibSpotFXData import ibFxPricesData
138+
ibfxpricedata = ibFxPricesData(conn)
139+
ibfxpricedata.get_list_of_fxcodes() # codes must be in .csv file /sysbrokers/IB/ibConfigSpotFx.csv
140+
ibfxpricedata.get_fx_prices("GBPUSD") # returns fxPrices object
141+
```
142+
143+
It's also possible to directly access the IB client for 'quick and dirty' work:
144+
145+
`conn.broker_get_fx_data("GBP", "USD")`
146+
147+
148+
<a name="futures_data_workflow"></a>
149+
# Reference
150+
151+
## Classes and object references
152+
153+
There are four types of objects in the [sysbrokers](/sysbrokers/) area of pysystemtrade:
154+
155+
- Client objects,
156+
- Server objects
157+
- Connection objects
158+
- Data source objects
159+
160+
### Client objects
161+
162+
Client objects make calls and requests to the broker.
163+
164+
#### Generic client object
165+
166+
The generic client object, [`brokerClient`](/sysbrokers/baseClient.py), contains a series of methods which need to be
167+
168+
These methods include:
169+
170+
- `broker_get_fx_data`: Returns an [`fxPrices`](/sysdata/fx/spotfx.py) object given a currency pair (NOTE: STRICTLY SPEAKING THIS SHOULD THEN BE WRAPPED in AN fxPricesData object...)
171+
172+
All methods are prefixed with `broker_` to reduce the chances of a conflict with the brokers own code.
173+
174+
175+
#### IB client object
176+
177+
The IB client object, [`ibClient`](/sysbrokers/IB/ibClient.py), inherits from `brokerClient`. It overrides the methods in `brokerClient`, and also contains any further methods required to interact with the IB Gateway to implement the `brokerClient` methods. All such methods are prefixed with `ib_` to distinguish them from `broker_` methods, and to reduce the chances of a conflict.
178+
179+
The overriden methods are:
180+
181+
- `broker_get_fx_data`: Returns an [`fxPrices`](/sysdata/fx/spotfx.py) object given a currency pair (NOTE: STRICTLY SPEAKING THIS SHOULD THEN BE WRAPPED in AN fxPricesData object...)
182+
183+
The extra methods include:
184+
185+
- `__init__`: Does the weird magical stuff required to get an IB client operating, and initialises the log, and request ID factory.
186+
- `ib_init_request_id_factory`, `ib_next_req_id`, `ib_clear_req_id`: make sure request ID's are parcelled out and shared nicely
187+
- `ib_resolve_spotfx_contract`, `ib_resolve_futures_contract`: Translate pysystemtrade depictions of an instrument into IB objects
188+
- `ib_resolve_contract`: Called by individual resolve methods, to go to IB and ensure we have the 'full' IB object.
189+
- `ib_get_historical_data`: Called by specific methods to get historical data
190+
191+
If you are going to connect to a new broker, then your main job is to ensure that you provide working methods to override those in `brokerClient`.
192+
193+
194+
### Server objects
195+
196+
Server objects handle data and messages coming back from the broker.
197+
198+
#### Generic server object
199+
200+
The generic server object, [`brokerServer`](/sysbrokers/baseServer.py) is very lightweight, and only contains methods for error capturing. We do not act on any errors, instead they are passed back to the client objects.
201+
202+
These methods include:
203+
204+
- `broker_init_error`: Prepare to receive errors. Should be called just before a connection is attempted to the broker (see connection objects below).
205+
- `broker_get_error`: Read the latest error from the queue
206+
- `broker_is_error`: True if any errors waiting to be read
207+
- `broker_error`: Add an error to the queue of errors.
208+
209+
All methods are prefixed with `broker_` to reduce the chances of a conflict with the brokers own code.
210+
211+
#### IB server object
212+
213+
The IB server object, [`ibServer`](/sysbrokers/IB/ibServer.py), inherits from `brokerServer` and from the IB `EWrapper`. It overrides the following `EWrapper` methods:
214+
215+
- `error`: Error report. Calls `broker_error` in `brokerServer` to handle the error
216+
- `contractDetails`, `contractDetailsEnd`: Receive contract details
217+
- `historicalData`, `historicalDataEnd`: Receive historical data
218+
219+
It also includes functions to help manage the queues which store data coming from IB to be passed to the client:
220+
221+
- `__init__`: Adds a log, and initialises dictionaries required to store queues.
222+
- `init_contractdetails`: prepare a queue for contract details
223+
- `init_historicprices`: prepare a queue for historic prices
224+
225+
### Connection objects
226+
227+
A connection object inherits from both a client and a server (NOTE: in the future we would need to add a data connection object to receive streamed prices, fills etc).
228+
229+
No generic connection object is provided, since they are likely to be highly bespoke to a particular broker. See [connectionIB](/sysbrokers/IB/ibConnection.py) for an example. At a minimum, connection objects must do the following:
230+
231+
- Logging: `def __init__(self, ..., log=logtoscreen())`. This also helpful: `log.label(broker="IB")`
232+
- Init the client and server objects, passing the log through
233+
- Init the error process `self.broker_init_error()`
234+
- Connect to the broker
235+
236+
Importantly the connection object will include methods that are inherited from [`brokerClient`](/sysbrokers/baseClient.py) and overriden in [`ibClient`](/sysbrokers/IB/ibClient.py). These are the main 'externally facing' methods.
237+
238+
### Data source objects
239+
240+
We treat IB as another data source, which means it has to conform to the data object API (see [storing futures and spot FX data](/docs/futures.md)).
241+
242+
#### Spot FX
243+
244+
For spot FX we have a class `ibFxPricesData` which inherits from the generic `fxPricesData`. This needs to be initialised with an IB connection:
245+
246+
247+
```
248+
from sysbrokers.IB.ibSpotFXData import ibFxPricesData
249+
ibfxpricedata = ibFxPricesData(conn)
250+
ibfxpricedata.get_list_of_fxcodes() # codes must be in .csv file /sysbrokers/IB/ibConfigSpotFx.csv
251+
ibfxpricedata.get_fx_prices("GBPUSD") # returns fxPrices object
252+
```
253+
254+
This is a 'read only' object; there are no methods implemented for writing or deleting FX data. Since connection objects abstract what the broker is doing, it should be possible to use `ibFxPricesData` for other brokers with minimal changes.
255+
256+
257+

‎docs/futures.md

+9-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,12 @@
1-
This document is specifically about *futures data*. It is broken into three sections. The first, [A futures data workflow](#futures_data_workflow), gives an overview of how data is typically processed. It describes how you would get some data from quandl, store it, and create back-adjusted prices. The next section [Storing futures data](#storing_futures_data) then describes in detail each of the components of the API for storing futures data. In the third and final section [simData objects](#simData_objects) you will see how we hook together individual data components to create a `simData` object that is used by the main simulation system.
1+
This document is specifically about storing and processsing *futures data*.
2+
3+
Related documents:
4+
5+
- [Using pysystemtrade as a production trading environment](/docs/production.md)
6+
- [Main user guide](/docs/userguide.md)
7+
- [Connecting pysystemtrade to interactive brokers](/docs/IB.md)
8+
9+
It is broken into three sections. The first, [A futures data workflow](#futures_data_workflow), gives an overview of how data is typically processed. It describes how you would get some data from quandl, store it, and create back-adjusted prices. The next section [Storing futures data](#storing_futures_data) then describes in detail each of the components of the API for storing futures data. In the third and final section [simData objects](#simData_objects) you will see how we hook together individual data components to create a `simData` object that is used by the main simulation system.
210

311
Although this document is about futures data, parts two and three are necessary reading if you are trying to create or modify any data objects.
412

0 commit comments

Comments
 (0)