Data Science - Bond Trading - Fixed Income Markets

Peter Simon

Managing Director, Financial Markets Data Science

Peter leads DataRobot’s financial markets data science practice and works closely with fintech, banking, and asset management clients on numerous high-ROI use cases for the DataRobot AI Platform.  

Prior to joining DataRobot, he gained twenty-five years’ experience in senior quantitative research, portfolio management, trading, risk management and data science roles at investment banks and asset managers including Morgan Stanley, Warburg Pincus, Goldman Sachs, Credit Suisse, Lansdowne Partners and Invesco, as well as spending several years as a partner at a start-up global equities hedge fund. Peter has an M.Sc. in Data Science from City, University of London, an MBA from Cranfield University School of Management, and a B.Sc. in Accounting and Financial Analysis from the University of Warwick.  His paper, “Hunting High and Low: Visualising Shifting Correlations in Financial Markets”, was published in the July 2018 issue of Computer Graphics Forum.

When Does Machine Learning Work Well in Financial Markets Blog 

A DATA SCIENTIST EXPLAINS: WHEN DOES MACHINE LEARNING WORK WELL IN FINANCIAL MARKETS?

January 17, 2023

by

Peter Simon

As a data scientist, one of the best things about working with DataRobot customers is the sheer variety of highly interesting questions that come up. Recently, a prospective customer asked me how I reconcile the fact that DataRobot has multiple very successful investment banks using DataRobot to enhance the P&L of their trading businesses with my comments that machine learning models aren’t always great at predicting financial asset prices. Peek into our conversation to learn when machine learning does—and doesn’t—work well in financial markets use cases.

https://www.datarobot.com/blog/a-data-scientist-explains-when-does-machine-learning-work-well-in-financial-markets/

Yield Book Analytics

Empowering financial markets with the best-in-class models, robust analytics, and related services.

Our brand is changing to LSEG Yield Book

As a trusted and authoritative source of fixed income analytics for more than 30 years, Yield Book offers an expanded set of capabilities that include market-leading data and cashflows modelling for in-depth security and portfolio analysis ranging from vanilla bonds to highly structured mortgages and complex derivatives for clients to comprehensively address their requirements. 

Why Yield Book?

Single source solution

Sophisticated analytics

Differentiating insights

Unrivalled flexibility

Regulatory alignment

Single Source Solution

Get access to the “golden source” of fixed income data, prices, indices, and analytics driven by Refinitiv’s Pricing and Reference data that underpins the construction of FTSE Russell indices. 

Industry-wide analytics solutions

Our market-leading models, derived analytics solutions and AI-powered workflows enable buy-side and sell-side customers to stay ahead of the competition. Our clients can access analytical solutions across the breadth of their business requirements, including:

Scenario aligned cashflows

Real-time user-specified parameters, such as curves 

Derived analytics solutions, such as SRI / FRTB analytics

Our solutions are underpinned by comprehensive asset class coverage, sophisticated market leading models, supported by world-class expertise in global financial markets. 

https://solutions.yieldbook.com/en

Machine learning-aided modeling of fixed income instruments

Daniel Martin, Barnabás Póczos

Machine Learning Department

Carnegie Mellon University

Pittsburgh, PA 15213

Burton Hollifield

Tepper School of Business

Carnegie Mellon University

Pittsburgh, PA 15213

https://www.ml.cmu.edu/research/dap-papers/f18/dap-martin-daniel.pdf

Supervised similarity learning for corporate bonds using

Random Forest proximities

Jerinsh Jeyapaulraj

2022

https://arxiv.org/pdf/2207.04368.pdf

BondGPT

2023

https://fortune.com/2023/06/13/bondgpt-artificial-intelligence-finance-bond-kings-bill-gross-jeffrey-gundlach/

PIMCO - UNDERSTANDING INVESTING in BONDS

https://europe.pimco.com/en-eu/resources/education/everything-you-need-to-know-about-bonds

data science in bond trading  - Google search interesting results

Handbook of Artificial Intelligence and Big Data Applications in Investments - CFA

 This book can be found at cfainstitute.org/ai-and-big-data

Download

Handbook of Artificial Intelligence and Big Data Applications in Investments  https://rpc.cfainstitute.org/en/research/foundation/2023/ai-and-big-data-in-investments-handbook

AI Pioneers in Investment Management

https://www.cfainstitute.org/-/media/documents/survey/AI-Pioneers-in-Investment-Management.ashx

Analytics for Grocery Retail Sales

https://www.mckinsey.com/industries/retail/our-insights/pushing-granular-decisions-through-analytics-in-na      

Personalized promotions through insights from analytics can be utilized by retailers. Retailers can operate the use cases at scale because technology has evolved, and customer touchpoints for data collection and communication (especially through e-grocery and loyalty apps) have increased in recent years. When done right, promotions can provide a substantial benefit of 4 to 8 percent sales increase and 2 to 3 percent net income and EBIT uplift.

 

What is Data Science? - Evolution of Data Science

Excerpts from Global Data Science Forum What is Data Science?
By Paco Nathan posted Mon March 04, 2019
https://community.ibm.com/community/user/datascience/blogs/paco-nathan/2019/03/04/what-is-data-science

What is Data Science?

A popular 2012 tweet by Josh Wills:

Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician.

Data Science gained traction in industry circa 2008, just as tooling for big data was on the rise, and as business use cases for machine learning (ML) became popularized. Those three grew together in contrast to an earlier era of business intelligence (BI), which was initially popularized by Gartner analyst Howard Dresner. Most of BI was defined atop data warehouse (DW) practices, based on work by Barry Devlin and Paul Murphy, Ralph Kimball, Bill Inmon, et al. BI and DW were both introduced in the late 1980s, then became widespread practices throughout the 1990s.

Data science emerged in response to demand for more advanced techniques and larger scale-out than what the best practices from the prior decade could provide. Cloud resources were becoming popular, and crucial insights could be obtained more quickly and more cost-effectively due to popular open source tools such as Hadoop, Spark, plus a whole range of Python libraries.

In 1962, a Bell Labs mathematician named John Tukey wrote a paper called “The Future of Data Analysis”.  Tukey urged a provocative new stance for applied mathematics which he called data analysis. The interesting section headings are:

“We should seek out wholly new questions to be answered.”
“We need to tackle old problems in more realistic frameworks.”
“We should seek out unfamiliar summaries of observational material, and establish their useful properties.”
“And still more novelty can come from finding, and evading, still deeper lying constraints.”

In the  books on visualizing data by Ed Tufte, references to Tukey show up throughout most all of books.

A generation later, another Bell Labs researcher named William Cleveland coined the term data science in a 2001 paper citing Tukey among others,  “Data science: An action plan for expanding the technical areas of the field of statistics”. Cleveland proposed an outline for a multi-disciplinary curriculum:

(25%) Multidisciplinary Investigations: data analysis collaborations in a collection of subject matter areas.
(20%) Models and Methods for Data: statistical models; methods of model building; methods of estimation and distribution based on probabilistic inference.
(15%) Computing with Data: hardware systems; software systems; computational algorithms.
(15%) Pedagogy: curriculum planning and approaches to teaching for elementary school, secondary school, college, graduate school, continuing education, and corporate training.
(5%) Tool Evaluation: surveys of tools in use in practice, surveys of perceived needs for new tools, and studies of the processes for developing new tools.
(20%) Theory: foundations of data science; general approaches to models and methods, to computing with data, to teaching, and to tool evaluation; mathematical investigations of models and methods, of computing with data, of teaching, and of evaluation.

This curriculum indicates what Cleveland thought the field required, namely that data science is a space in which statistics and computing needed to interact, to provide the necessary resources and scale.

That same year, a UC Berkeley professor named Leo Breiman wrote “Statistical Modeling: The Two Cultures”. One culature is of the previous era which he called data modeling and a new trend emerging which he called algorithmic modeling. That culture of data modeling was what Tukey had argued against.  The newer culture embraced much larger data rates and more computation and also leveraged machine learning algorithms to help automate decisions at scale.

The current heyday of data science began when some of these applications which required more data started to become tractable, reliable, and cost-effective (in that order).

Check out these histories by lead architects at those firms – roughly centered on Q3 1997, which turned out to be a key inflection point for the Dot Com Boom:

“Early Amazon: Splitting the website”, Greg Linden, Amazon
“eBay Architecture”, Randy Shoup, eBay
“Inktomi’s Wild Ride”, Erik Brewer, Yahoo! Search (0:05:31 ff)
“Underneath the Covers at Google”, Jeff Dean, Google (0:06:54 ff)

The timing for those projects was during the peak of data warehouses and business intelligence adoption. However, a common theme among those four architects’ reflections is that they recognized how they’d need to scale ecommerce applications but could not do so with available tooling. Instead they turned to open source tools (such as Linux) for early data science work on proto clouds, leveraging ML at scale for ecommerce. Their timing was impeccable, particularly for Amazon: just in time to monetize the first big wave of ecommerce in the holiday season of Q4 1997. The rest is history.

The gist is that ecommerce firms split their web apps using a principle of horizontal scale out, i.e., proto cloud work on server farms. Those many servers generated lots of log files (proto Big Data), which in turn were analyzed using machine learning algorithms, which in turn provided predictive analytics that improved customer experience in the web apps. A virtuous cycle emerge, with data as a product.

However, after Q4 1997 the world of data changed, predictive analytics loomed large. Breiman described that sea change quite succinctly:

A new research community using these tools sprang up. Their goal was predictive accuracy. The community consisted of young computer scientists, physicists and engineers plus a few aging statisticians. They began using the new tools in working on complex prediction problems where it was obvious that data models were not applicable: speech recognition, image recognition, nonlinear time series prediction, handwriting recognition, prediction in financial markets.

Plenty of other people also helped further the cause of “data science” and deserve credit, such as Jeff Wu who likely coined the phrase (in its contemporary usage) during his U Michigan appointment lecture “Statistics = Data Science?”

The main takeaway from this article:

Looking at decades of history, data science found its place by applying increasingly advanced mathematics for novel business cases, in response to surges in data rates and compute resources.

In the latest wave of AI applications in industry, we have the term ABC emerging to describe a winning combo of “AI”, “Big Data”, and “Cloud Computing” – as the latest embodiment of that takeaway described above.

Beyond the well-known roles of data scientist and data engineer, there’s another important role emerging which has not yet been named. We found that 23% of the enterprise organizations attempting to leverage data science, machine learning, artificial intelligence, etc., cite recognize business use case as a critically missing skill within their teams. What would you call that role? Where and how does a person learn to perform it?

Data Science - More explanations

Data Science in Manufacturing and Automation

https://www.slideshare.net/RavishankarRajagopal/data-science-in-manufacturing-and-automation

What is Data Science?

 In a 2009 McKinsey&Company article, Hal Varian, Google's chief economist and UC Berkeley professor of information sciences, business, and economics, predicted the importance of adapting to technology’s influence and reconfiguration of different industries. 2

“The ability to take data — to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it — that’s going to be a hugely important skill in the next decades.”
https://datascience.berkeley.edu/about/what-is-data-science/

Chapter 1. Introduction: What Is Data Science?
Doing Data Science by Rachel Schutt, Cathy O'Neil
https://www.oreilly.com/library/view/doing-data-science/9781449363871/ch01.html

Data Science vs. Big Data vs. Data Analytics
By Shivam Arora
Jan 4, 2019
https://www.simplilearn.com/data-science-vs-big-data-vs-data-analytics-article

Oracle Artificial Intelligence (AI)—What Is Data Science?

An elaborate article giving many details of data science and related issues.

https://www.oracle.com/in/artificial-intelligence/what-is-data-science.html

Earlier Articles

What is Data Science? - An Introduction to Data Science

Data Science - Online Study Notes and Video Courses - Free Also

Data Analytics and Data Mining - Difference Explained


Updated on 10.7.2022,  31 May 2019, 26 May 2019

Marketing - Analytics, AI, Data Science and Machine Learning

Browse  Online MBA Management Theory Handbook

Marketing the Future: How Data Analytics Is Changing

Nov 23, 2020

Data analytics helps marketers learn about their customers with target precision, from the movies they watch on Netflix to their favorite scoop of chocolate ice cream.

https://knowledge.wharton.upenn.edu/article/marketing-future-data-analytics-changing/

[Webinar] Marketing Analytics in the Data-driven World

28 Aug 2020

https://www.youtube.com/watch?v=urVSPyVAtX0

Top 8 Data Science Use Cases in Marketing

2019

https://www.kdnuggets.com/2019/11/top-8-data-science-use-cases-marketing.html

ABCD of Machine Learning for Marketing

Are you interested in learning how organizations are increasing sales using machine learning in an intuitive and fun way? Then this ten-part case study is for you. Enjoy learning!

Part 1: Marketing analytics and customer behavior and  psychology for starters https://lnkd.in/bax6ZCf

Part 2: Return on marketing investment (ROMI) https://lnkd.in/fKUKput

Part 3: Exploratory data analysis to grow revenue https://lnkd.in/fRzT4D2

Part 4: Keep an eye on your customer's market basket https://lnkd.in/fibiC67

Association Analysis

There are a few association analysis metrics (i.e. support, confidence, and lift) that are really helpful in deciphering information hidden in the dataset of purchases of various products.

Part 5: Who will buy my product? Decision trees for starters https://lnkd.in/fXa2U_R

Classification and Regression Tree (CART)

Part 6: Decision tree details https://lnkd.in/bNQtVRp

Part 7: Choose the best model to optimize sales https://lnkd.in/f2AFPhg

Part 8: How machines decide? Artificial neural networks for starters https://lnkd.in/f2AFPhg

Part 9: But tell me how much money I will earn by sales? Regression for starters https://lnkd.in/fcGHQbH

Part 10: Aha! Time to grow sales. https://lnkd.in/b5AYrzR

Information identified in Linkedin Community of IITBombay.

Marketing Analytics - E-Book - McKinsey

117 pages 2015 Book
https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/Marketing%20and%20Sales/Our%20Insights/EBook%20Big%20data%20analytics%20and%20the%20future%20of%20marketing%20sales/Big-Data-eBook.ashx


Marketing Analytics: What it is and why it matters

Marketing Analytics Solutions from SAS

SAS® Marketing Automation
Get more campaigns out the door in an automated, trackable and highly repeatable fashion.
Customer Intelligence 360
Infuse your marketing decisions with unprecedented customer insights, and create relevant, satisfying, valued customer experiences.
SAS® Marketing Optimization
Make the most of each customer contact by determining how business variables will affect outcomes.
https://www.sas.com/en_us/insights/marketing/marketing-analytics.html

Using marketing analytics to drive superior growth

June 2014
https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/using-marketing-analytics-to-drive-superior-growth

Book: Marketing Analytics: Strategic Models and Metrics

Stephan Sorger
CreateSpace Independent Publishing Platform, 31-Jan-2013 - Business & Economics - 488 pages

Chapter 1. Introduction - Introduction to marketing analytics
Chapter 2. Market Insight - Market sizing and trend analysis
Chapter 3. Market Segmentation - Segment identification, analysis, and strategy
Chapter 4. Competitive Analysis - Competitor identification, analysis, and strategy
Chapter 5. Business Strategy - Analytics-based strategy selection
Chapter 6. Business Operations - Forecasting, predictive analytics, and data mining
Chapter 7. Product and Service Analytics - Conjoint analysis and product/service metrics
Chapter 8. Price Analytics - Pricing techniques and assessment
Chapter 9. Distribution Analytics - Analytics-based channel evaluation and selection
Chapter 10. Promotion Analytics - Promotion budget estimation and allocation
Chapter 11. Sales Analytics - Metrics for sales, profitability, and support
Chapter 12. Analytics in Action - Pivot tables and data-driven presentations

Interview with Stephan Sorger
https://www.regalix.com/insights/future-business-age-marketing-analytics/

http://www.stephansorger.com/blog.html

https://voices.berkeley.edu/business/unlocking-marketing-data-analytics

Marketing Analytics - Google Books

Data Science for Marketing Analytics: Achieve your marketing goals
https://books.google.co.in/books?id=s86PDwAAQBAJ
Tommy Blanchard, ‎Debasish Behera, ‎Pranshu Bhatnagar - 2019 - ‎Preview 
What you will learn Analyze and visualize data in Python using pandas and Matplotlib Study clustering techniques, such as hierarchical and k-means clustering Create customer segments based on manipulated data Predict customer lifetime value etc.


Digital Marketing Analytics: Making Sense of Consumer Data
https://books.google.co.in/books?isbn=0134998650
Chuck Hemann, ‎Ken Burbary - 2018 - ‎Preview - ‎
Because you’ve barely begun to use it, that’s why! Good news: neither have your competitors. It’s hard! But digital marketing analytics is 100% doable, it offers colossal opportunities, and all of the data is accessible to you.

Marketing Analytics: A Practical Guide ..
https://books.google.co.in/books?isbn=0749482176
Mike Grigsby - 2018 - ‎Preview - ‎
Complete with downloadable data sets and test bank resources, this book supplies a concrete foundation to optimize marketing analytics for day-to-day business advantage.

Marketing Analytics: A Practical Guide to Improving Consumer
https://books.google.co.in/books?isbn=0749482176
Mike Grigsby - 2018 - ‎Preview - ‎
'With its focus on practicality, this book is an invaluable toolkit of frameworks to drive consumer-centric analytics initiatives across marketing organizations.

Handbook of Marketing Analytics: Methods and Applications
https://books.google.co.in/books?isbn=1784716758
Natalie Mizik, ‎Dominique M. Hanssens - 2018 - ‎Preview - ‎
The Handbook of Marketing Analytics showcases the analytical methods used in marketing and their high-impact real-life applications.

Predictive Analytics for Marketers: Using Data Mining
https://books.google.co.in/books?isbn=0749479949
Barry Leventhal - 2018 - ‎Preview - ‎
Including comprehensive coverage of an array of predictive analytic tools and techniques, this book enables readers to harness patterns from past data, to make accurate and useful predictions that can be converted to business success.


Mastering Market Analytics: Business Metrics – Practice and Application
https://books.google.co.in/books?isbn=1787148351
Robert Kozielski - 2017 - ‎Preview - ‎
In Mastering Market Analytics, Robert Kozielski presents different measurement systems and marketing activities, along with common mistakes made by organizations and managers in the process of building measurement, and illustrates how to ...



New Methods of Market Research and Analysis
https://books.google.co.in/books?isbn=1786432692
G. Scott Erickson - 2017 - ‎Preview - ‎
This book can be used as a supplement to a traditional marketing research text or on its own.


Digital Analytics for Marketing
https://books.google.co.in/books?isbn=1317278437
Marshall Sponder, ‎Gohar F. Khan - 2017 - ‎Preview - ‎
This comprehensive book provides students with a "grand tour" of the tools needed to measure digital activity and implement best practices for using data to inform marketing strategy.



Handbook of Marketing Decision Models
https://books.google.co.in/books?isbn=3319569414

Berend Wierenga, ‎Ralf van der Lans - 2017 - ‎Preview - ‎
The Second Edition of this book presents the state of the art in this important field.

Principles of Marketing Engineering and Analytics, 3rd Edition
https://books.google.co.in/books?isbn=098576483X
Gary L. Lilien, ‎Arvind Rangaswamy, ‎Arnaud De Bruyn - 2017 - ‎Preview
We have designed this book primarily for the business school student or marketing manager, who, with minimal background and technical training, must understand and employ the basic tools and models associated with Marketing Engineering.

Marketing at the Confluence between Entertainment and Analytics
https://books.google.co.in/books?isbn=331947331X
Patricia Rossi - 2017 - ‎Preview 
This volume presents the full proceedings of the 2016 Academy of Marketing Science (AMS) World Marketing Congress held in Paris, France.

Marketing Strategy: Based on First Principles and Data Analytics
https://books.google.co.in/books?isbn=1137526246
Robert W. Palmatier, ‎Shrihari Sridhar - 2017 - ‎Preview - ‎
A brand new textbook with an innovative and exciting approach to marketing strategy.


Updated on 19 July 2021,  5 June 2019, 1 June 2019

Manufacturing and Manufacturing Management Analytics - Introduction and Bibliography

Manufacturing Analytics for problem-solving processes in production

Maximilian Meister et al.
Procedia CIRP
Volume 81, 2019, Pages 1-6
Open Access Article
https://www.sciencedirect.com/science/article/pii/S2212827119303051


4 September 2019
Making Sensors Bug Proof - Ford
https://www.cnet.com/roadshow/news/ford-self-driving-car-sensor-maintenance/

AI-powered analytics for Manufacturing - Solutions for Manufacturing

while there is significant value in the data Manufacturing companies produce, both structured and unstructured, too little of it is being analyzed.

To turn this information into better, smarter, and faster decision making, manufacturers must be able to fully exploit the mountains of data they produce. Through AI and analytics, companies can increase operational productivity, gain a competitive advantage and develop new business opportunities.

OpenText AI and Analytics
Using OpenText™ Magellan™ Analytics Suite and OpenText™ Magellan™, the AI-powered machine learning platform, Manufacturing companies can apply predictive algorithms to big data from both internal and external sources to generate accurate predictions and make better decisions.
https://www.opentext.com/products-and-solutions/products/ai-and-analytics/analytics-manufacturing


2 July 2019
Armed With Analytics: Manufacturing as a Martial Art
https://www.industryweek.com/leadership/armed-analytics-manufacturing-martial-art

June 2019

Manufacturing Analytics: Colgate Palmolive Optimizes for Agility and Productivity

Manufacturing operations have to support growth and agility while continuing to excel at operational efficiency. Ann Tracy, VP Global EHS, Sustainability and Supply Chain Strategy Colgate-Palmolive

The key enablers for productivity in the manufacturing network have to be identified. Colgate has blended design and manufacturing analytics to accelerate the pace of decision making. The initiative has achieved scale  and supports continuous improvement with user engagement and ownership.
https://www.gartner.com/en/conferences/emea/supply-chain-spain/speakers/case-study




1 Jan 2019

Benefits of Manufacturing Analytics
https://blog.aimultiple.com/manufacturing-analytics/

November 2018

How a German Manufacturing Company Set Up Its Analytics Lab

Niklas GobyTobias BrandtDirk Neumann

Three years on, ZF Data Lab is a valuable addition to the company.  ZF has been able to solve problems that had stumped the company’s engineers for years using ZF Data Lab Two examples were given in HBR article.
https://hbr.org/2018/11/how-a-german-manufacturing-company-set-up-its-analytics-lab



2 Oct 2018

Machine Manufacturing Analytics – Contributing to the Evolution of an Industry

SAP Machine Manufacturing Analytics visualizes a large set of real-time data through a newly designed interface, which allows engineers to adjust production processes as they are happening, and to make better predictions moving forward. Ultimately, SAP Machine Manufacturing Analytics helps design manufacturing processes and is scalable to monitor large cereal production in multiple locations.
https://experience.sap.com/news/machine-manufacturing-analytics-contributing-to-the-evolution-of-an-industry/

Types of Data Scientists and Organizing Data Science and Analytics Department

To hire the right people for the right roles and organize the data science department, it’s important to distinguish between different types of data scientist.

One type of data scientist creates output for the decision makers to use  in the form of product and strategy recommendations. They are decision scientists. The other creates output for using on machines like models, training data, and algorithms. They are modeling scientists.

Five key areas are required for  data science operations. In small organizations, one person may  do several of these things. In slightly bigger teams, each of these may be a role staffed by one or more individuals. In larger operations, each may be a team unto itself. These roles cover the creation, maintenance, and use of data.

Data infrastructure: data ingestion, availability, operations, access, and running environments to support workflows of data scientists. e.g. running a Hadoop cluster

Data engineering: determination of data schemas needed to support measurement and modeling needs, and data cleansing, aggregation, ETL, dataset management

Data quality and data governance: tools, processes, guidelines to ensure data is correct, gated and monitored, documented, standardized. This includes tools for data lineage and data security.

Data analytics engineering:  analytics software libraries, productizing workflows, and analytic microservices.

Data-product product manager: creating products for internal customers to use within their workflow, to enable incorporation of measurements and outputs created by data scientists. Examples include: a portal to read out results of A/B tests, a failure analysis tool, or a dashboard that enables self serve data and root cause diagnosing of changes to metrics or model performance.

How to organize Data Science Department?

Source
The Kinds of Data Scientist
Yael Garten
HBR, NOVEMBER 2018
https://hbr.org/2018/11/the-kinds-of-data-scientist