Converting IbPy Data Request to Pandas DataFrame

Introduction

Interactive Brokers (IB) provides an API for financial institutions and traders to access its markets through various programming languages. The ib.ext.Contract class is used to define the contract, which specifies the symbol, exchange, currency, and expiration date of the instrument being requested. In this article, we will explore how to convert IB’s data request into a pandas DataFrame, bypassing the need for CSV files.

Understanding IbPy API

IBPy is a Python wrapper for the Interactive Brokers API. It allows developers to access the API using a Python interface, which simplifies the process of connecting to the platform and retrieving historical market data. The ib.ext.Contract class is used to define the contract, specifying the symbol, exchange, currency, and expiration date.

Historical Data Request

To retrieve historical market data, you need to send a request to the IB API using the reqHistoricalData method. This method requires several parameters:

• symbol: The instrument being requested (e.g., ES for E-mini S&P 500 futures)
• secType: The security type (e.g., FUT for futures)
• exchange: The exchange on which the instrument is traded
• currency: The currency in which the instrument is denominated
• expiry: The expiration date of the instrument
• barCount: The number of bars to retrieve (in this case, 1 bar per second for 1 day)
• period: The period over which the data is retrieved
• stream: Whether to stream data in real-time or not

In our example, we are requesting historical market data for E-mini S&P 500 futures (ES) with an expiration date of September 9th.

Modifying Historical Data Handler Function

To convert the IBPy data request into a pandas DataFrame, you need to modify the historical_data_handler function. This function is called whenever new data arrives from the IB API and handles it accordingly.

The key modification in our example is the following code:

# Modified historical data handler function
else:
    df_new = pd.DataFrame(columns=['Code', 'Date', 'Close'])
    j = 0
    for i in newDataList:
        # Split each row into its constituent parts
        df_new.loc[j] = i.split(',')
        j = j + 1
    # Set the Date column as the index of the DataFrame
    df_new.set_index('Date', inplace=True)

This modification creates a new pandas DataFrame df_new with three columns: Code, Date, and Close. It then populates this DataFrame by splitting each row in the newDataList into its constituent parts using the comma (,) as a delimiter.

Alternative Approach: Writing Values Directly to DataFrame

Another approach is to write the values directly into the DataFrame instead of first placing them in a string and splitting. This can be achieved by modifying the code as follows:

# Modified historical data handler function with alternative approach
else:
    # Create a new pandas DataFrame
    df_new = pd.DataFrame(columns=['Code', 'Date', 'Close'])
    # Write values directly into the DataFrame
    for i in newDataList:
        close_value = msg.close  # Get the close value from the message
        row_values = [new_symbol, strftime("%Y-%m-%d", localtime(int(msg.date))), close_value]
        df_new.loc[len(df_new)] = row_values

This modification avoids creating a string and splitting it into constituent parts. Instead, it writes the values directly into the DataFrame.

Setting Date Column as Index

To set the Date column as the index of the DataFrame, you can use the set_index method:

# Set the Date column as the index of the DataFrame
df_new.set_index('Date', inplace=True)

This modification ensures that the data is stored in a convenient and efficient format.

Example Use Case

Here’s an example of how to use this approach to retrieve historical market data for E-mini S&P 500 futures (ES) with an expiration date of September 9th:

# Define the contract and IB API connection
new_symbol = 'ES'
qqq = Contract()
qqq.m_symbol = new_symbol
qqq.m_secType = 'FUT'
qqq.m_exchange = 'GLOBEX'
qqq.m_currency = 'USD'
qqq.m_expiry = '201609'

con = ibConnection()
con.register(historical_data_handler, message.historicalData)
con.connect()

# Retrieve historical market data for 1 bar per second for 1 day
symbol_id = 0
for i in new_symbolinput:
    qqq.m_symbol = i
    con.reqHistoricalData(symbol_id, qqq, '', '1 W', '1 day', 'TRADES', 1, 2)

This example demonstrates how to retrieve historical market data for E-mini S&P 500 futures (ES) with an expiration date of September 9th using the IB API. The historical_data_handler function is then modified to convert this data into a pandas DataFrame.

Conclusion

Converting IBPy’s data request into a pandas DataFrame bypasses the need for CSV files, making it more efficient and convenient to work with market data. By modifying the historical_data_handler function to write values directly into the DataFrame, you can avoid creating strings and splitting them into constituent parts. The resulting DataFrame is then easy to manipulate and analyze using various pandas functions.

I hope this article has provided a comprehensive overview of how to convert IBPy’s data request into a pandas DataFrame. If you have any questions or need further clarification on any of the concepts discussed, please feel free to ask!

Last modified on 2024-01-12