Strategic Insights: Analyzing S&P500 with ISM PMI in Python
Written on
Chapter 1: Understanding Sentiment Analysis in Trading
Sentiment analysis represents an expansive and promising area within data analytics and trading. This form of analysis has seen rapid growth, employing current market sentiments to gauge the intentions and positions of participants.
Consider planning to watch a movie and seeking recommendations from friends who have already seen it. If 75% of them express positive feedback, you might feel confident that you'll enjoy it, as the prevailing sentiment is favorable (ignoring personal preferences). A similar approach can be applied to financial markets through various quantitative and qualitative methods.
Indicators can often serve multiple purposes; a technical indicator might also function as a sentiment indicator (e.g., On-Balance Volume). Our analyses can be technical (like drawing support and resistance lines) or quantitative (such as mean-reversion strategies).
This article will delve into the Institute for Supply Management's Purchasing Manager's Index (PMI) and its application in forecasting trends for the S&P500. We will evaluate signal quality as a key metric.
The Fibonacci Trading Book is now available! This resource contains Fibonacci-based trading methods (tools, indicators, patterns, and strategies) designed to enhance your trading and analytical skills by integrating this crucial technical analysis approach. [PDF Version available at the end of this article.]
Section 1.1: The ISM PMI Explained
Sentiment analysis functions as a predictive analytics tool that leverages alternative data, beyond the fundamental OHLC price structure. It often relies on subjective polls and scores but can also incorporate quantitative measures, such as anticipated hedges informed by market strength or weakness.
The Institute for Supply Management conducts a monthly survey known as the Purchasing Manager's Index (PMI), querying 400 representatives from industrial firms about their current and projected business activities. The PMI comprises five components that can also be analyzed independently:
- New Orders — 30% weight.
- Production — 25% weight.
- Employment — 20% weight.
- Supplier Deliveries — 15% weight.
- Inventories — 10% weight.
Typically, a PMI reading of 50 indicates a neutral economic state (no growth or contraction). A score above 50 signals manufacturing expansion, while a score below 50 indicates contraction. The PMI tends to peak before the economy does, leading by 6–12 months, making it a remarkable leading indicator (with a correlation of over 0.70 with US GDP).
Historically, PMI values have ranged between 30 and 60, functioning as signals for potential reversals. The objective of our analysis is to validate this by developing a strategy that generates directional recommendations for the S&P500 based on mean-reversion principles.
Section 1.2: Data Acquisition and Strategy Development
Let’s begin by downloading historical data for both the ISM PMI and the S&P500, dating back to 1970. I've compiled this data into an Excel file available for download from my GitHub. We can easily import this data into the Python interpreter for analysis.
First, download the ISM_PMI.xlsx file to your computer, then navigate to the Python interpreter and set the source to the location of the Excel file. The next step is to import the data and structure it accordingly.
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
Defining Core Functions:
def adder(Data, times):
for i in range(1, times + 1):
new = np.zeros((len(Data), 1), dtype=float)
Data = np.append(Data, new, axis=1)
return Data
def deleter(Data, index, times):
for i in range(1, times + 1):
Data = np.delete(Data, index, axis=1)return Data
def jump(Data, jump):
Data = Data[jump:, ]
return Data
def indicator_plot_double(Data, first_panel, second_panel, window=250):
fig, ax = plt.subplots(2, figsize=(10, 5))
ax[0].plot(Data[-window:, first_panel], color='black', linewidth=1)
ax[0].grid()
ax[1].plot(Data[-window:, second_panel], color='brown', linewidth=1)
ax[1].grid()
ax[1].legend()
We should now have an array called my_data containing historical data for both the S&P500 and the PMI since 1970, with the S&P500 data in the first panel and the ISM PMI in the second.
Subjectively, we can establish a barrier at 40 as a support level for the economy and a resistance at 60, which will serve as our trading signals for buy and sell decisions. While some hindsight bias may be present, our goal is not to create a past trading strategy but to assess if these barriers will remain effective in the future. Based on this intuition, the following trading rules can be established:
- Long (Buy) the stock market whenever the ISM PMI hits 40.
- Short (Sell) the stock market whenever the ISM PMI reaches 60.
def signal(Data, ism_pmi, buy, sell):
Data = adder(Data, 2)
for i in range(len(Data)):
if Data[i, ism_pmi] <= 40 and all(Data[i - j, buy] == 0 for j in range(1, 6)):
Data[i, buy] = 1elif Data[i, ism_pmi] >= 60 and all(Data[i - j, sell] == 0 for j in range(1, 6)):
Data[i, sell] = -1return Data
This code generates signals represented in the chart. The ISM PMI serves as a buying-the-dips indicator, although it may require adjustments to enhance signal accuracy. We can assess this using a single performance metric: signal quality.
Evaluating Signal Quality:
Signal quality measures the market's reaction after a specified time period following a signal. This metric assumes a fixed holding period, allowing us to compare the market level at that point to the entry level.
# Choosing a period
period = 21
def signal_quality(Data, closing, buy, sell, period, where):
Data = adder(Data, 1)
for i in range(len(Data)):
try:
if Data[i, buy] == 1:
Data[i + period, where] = Data[i + period, closing] - Data[i, closing]if Data[i, sell] == -1:
Data[i + period, where] = Data[i, closing] - Data[i + period, closing]except IndexError:
passreturn Data
# Using a Window of Signal Quality Check
my_data = signal_quality(my_data, 1, 2, 3, period, 4)
positives = my_data[my_data[:, 4] > 0]
negatives = my_data[my_data[:, 4] < 0]
# Calculating Signal Quality
signal_quality = len(positives) / (len(negatives) + len(positives))
print('Signal Quality = ', round(signal_quality * 100, 2), '%')
The signal quality of 70.83% suggests that out of 100 trades, approximately 71 are likely to yield profitable outcomes, excluding transaction costs. Although 24 signals were generated under the provided conditions, these results are somewhat disappointing for an indicator claimed to be a leader. However, there remains potential for enhancement through various optimization techniques:
- Introduce a lag factor for the S&P500 to assess historical value.
- Consider alternative strategies based on surpassing or breaking the 50 level.
- Adjust the barriers from 40 and 60 to 35 and 65 to evaluate any improvements.
- Explore strategies focused on exiting barriers rather than entering them.
- Optimize the holding period to identify the best reaction window.
- Modify entry and exit conditions by incorporating risk management and entry optimization, such as closing positions prior to entering new ones.
For a detailed exploration of Fibonacci-based trading methods, check out the PDF link for the Fibonacci Trading Book (ensure to include your email address in the notes section).
If you're interested in learning how to develop various algorithms independently, I highly recommend checking out Lumiwealth. They offer comprehensive courses on algorithmic trading, blockchain, and machine learning.