https://books.google.co.in/books?id=absoBgAAQBAJ
http://www.springer.com/us/book/9783319135021
Authors: Lake, Peter, Drake, Robert
Publisher Springer
Table of Contents
1 Introducing Big Data .............................................................................. 1
1.1 What the Reader Will Learn......................................................... 1
1.2 Big Data: So What Is All the Fuss About?................................... 1
1.2.1 Defining “Big Data”...................................................... 2
1.2.2 Big Data: Behind the Hype ........................................... 3
1.2.3 Google and a Case of the Flu........................................ 6
1.3 Big Data: The Backlash Begins ................................................... 6
1.3.1 Big Data Catches a Cold............................................... 6
1.3.2 It’s My Data – So What’s in It for Me?......................... 8
1.3.3 Bucking the Backlash – The Hype Cycle ..................... 9
1.4 A Model for Big Data .................................................................. 11
1.4.1 Strategy ......................................................................... 12
1.4.2 Structure........................................................................ 13
1.4.3 Style .............................................................................. 13
1.4.4 Staff............................................................................... 13
1.4.5 Statistical Thinking ....................................................... 14
1.4.6 Synthesis....................................................................... 14
1.4.7 Systems......................................................................... 15
1.4.8 Sources.......................................................................... 15
1.4.9 Security ......................................................................... 16
1.5 Summary ...................................................................................... 16
1.6 Review Questions......................................................................... 16
1.7 Group Work/Research Activity .................................................... 17
1.7.1 Discussion Topic 1 ........................................................ 17
1.7.2 Discussion Topic 2 ........................................................ 17
References................................................................................................. 17
2 Strategy .................................................................................................... 19
2.1 What the Reader Will Learn......................................................... 19
2.2 Introduction.................................................................................. 19
2.3 What is Strategy? ......................................................................... 20
2.4 Strategy and ‘Big Data’................................................................ 22
2.5 Strategic Analysis......................................................................... 26
2.5.1 Analysing the Business Environment ........................... 26
2.5.2 Strategic Capability – The Value Chain ........................ 32
2.5.3 The SWOT ‘Analysis’................................................... 38
2.6 Strategic Choice ........................................................................... 43
2.6.1 Introduction................................................................... 43
2.6.2 Type, Direction and Criteria
of Strategic Development.............................................. 44
2.6.3 Aligning Business and IT/IS Strategy........................... 47
2.7 Summary ...................................................................................... 51
2.8 Review Questions......................................................................... 51
2.9 Group Work Research Activities.................................................. 51
2.9.1 Discussion Topic 1 ........................................................ 51
2.9.2 Discussion Topic 2 ........................................................ 51
References................................................................................................. 52
3 Structure .................................................................................................. 53
3.1 What the Reader Will Learn......................................................... 53
3.2 Introduction.................................................................................. 53
3.3 What Is ‘Structure’? ..................................................................... 55
3.3.1 What Do We Mean by ‘Structure’?............................... 55
3.4 Formal Structures......................................................................... 56
3.4.1 The “Organisational Chart”: What Does
It Tell Us?...................................................................... 56
3.4.2 Structure, Systems and Processes................................. 61
3.4.3 Formal Structure: What Does This Mean
for Big Data?................................................................. 65
3.4.4 Information Politics ...................................................... 66
3.5 Organisational Culture: The Informal Structure .......................... 69
3.5.1 What Do We Mean by ‘Culture’? ................................. 69
3.5.2 Culture and Leadership................................................. 69
3.5.3 The “Cultural Web”....................................................... 71
3.6 Summary ...................................................................................... 78
3.7 Review Questions......................................................................... 78
3.8 Group Work/Research Activity .................................................... 78
3.8.1 Discussion Topic 1 ........................................................ 78
3.8.2 Discussion Topic 2 ........................................................ 79
References................................................................................................. 79
4 Style .......................................................................................................... 81
4.1 What the Reader Will Learn......................................................... 81
4.2 Introduction.................................................................................. 81
4.3 Management in the Big Data Era................................................. 82
4.3.1 Management or Leadership?......................................... 82
4.3.2 What Is ‘Management’?................................................ 83
4.3.3 Styles of Management................................................... 86
4.3.4 Sources of Managerial Power ....................................... 87
4.4 The Challenges of Big Data (the Four Ds)................................... 90
4.4.1 Data Literacy................................................................. 90
4.4.2 Domain Knowledge ...................................................... 92
4.4.3 Decision-Making........................................................... 93
4.4.4 Data Scientists............................................................... 96
4.4.5 The Leadership Imperative ........................................... 98
4.5 Summary ...................................................................................... 99
4.6 Review Questions......................................................................... 100
4.7 Group Work/Research Activity .................................................... 100
4.7.1 Discussion Topic 1 ........................................................ 100
4.7.2 Discussion Topic 2 ........................................................ 100
References................................................................................................. 100
5 Staff .......................................................................................................... 103
5.1 What the Reader Will Learn......................................................... 103
5.2 Introduction.................................................................................. 103
5.3 Data Scientists: The Myth of the ‘Super Quant’.......................... 104
5.3.1 What’s in a Name? ........................................................ 104
5.3.2 Data “Science” and Data “Scientists”........................... 105
5.3.3 We’ve Been Here Before…........................................... 110
5.4 It Takes a Team…......................................................................... 111
5.4.1 What Do We Mean by “a Team”?................................. 111
5.4.2 Building High-Performance Teams .............................. 113
5.5 Team Building as an Organisational Competency ....................... 117
5.6 Summary ...................................................................................... 121
5.7 Review Questions......................................................................... 121
5.8 Group Work/Research Activity .................................................... 122
5.8.1 Discussion Topic 1 ........................................................ 122
5.8.2 Discussion Topic 2 ........................................................ 122
References................................................................................................. 122
6 Statistical Thinking................................................................................. 125
6.1 What the Reader Will Learn......................................................... 125
6.2 Introduction: Statistics Without Mathematics.............................. 125
6.3 Does “Big Data” Mean “Big Knowledge”? ................................. 126
6.3.1 The DIKW Hierarchy ................................................... 127
6.3.2 The Agent-in-the-World................................................ 128
6.4 Statistical Thinking – Introducing System 1 and System 2 ......... 129
6.4.1 Short Circuiting Rationality.......................................... 132
6.5 Causality, Correlation and Conclusions....................................... 133
6.6 Randomness, Uncertainty and the Search for Meaning............... 135
6.6.1 Sampling, Probability and the Law
of Small Numbers......................................................... 137
6.7 Biases, Heuristics and Their Implications for Judgement............ 138
6.7.1 Non-heuristic Biases..................................................... 142
6.8 Summary ...................................................................................... 144
6.9 Review Questions......................................................................... 144
6.10 Group Work Research Activities.................................................. 145
6.10.1 Discussion Topic 1 – The Linda Problem..................... 145
6.10.2 Discussion Topic 2 – The Birthday Paradox................. 145
References................................................................................................. 146
7 Synthesis................................................................................................... 147
7.1 What the Reader Will Learn......................................................... 147
7.2 From Strategy to Successful Information Systems...................... 147
7.2.1 The Role of the Chief Information Officer (CIO)......... 148
7.2.2 Management of IS Projects........................................... 149
7.3 Creating Requirements That Lead to Successful
Information Systems.................................................................... 151
7.4 Stakeholder Buy-In ...................................................................... 154
7.5 How Do We Measure Success...................................................... 155
7.6 Managing Change ........................................................................ 156
7.7 Cost Benefits and Total Cost of Ownership ................................. 158
7.7.1 Open Source.................................................................. 158
7.7.2 Off the Shelf vs Bespoke .............................................. 159
7.7.3 Gauging Benefits........................................................... 160
7.8 Insourcing or Outsourcing?.......................................................... 161
7.9 The Effect of Cloud...................................................................... 163
7.10 Implementing ‘Big Data’.............................................................. 164
7.11 Summary ...................................................................................... 165
7.12 Review Questions......................................................................... 166
7.13 Group Work Research Activities.................................................. 166
7.13.1 Discussion Topic 1 ........................................................ 166
7.13.2 Discussion Topic 2 ........................................................ 166
References................................................................................................. 166
8 Systems..................................................................................................... 169
8.1 What the Reader Will Learn......................................................... 169
8.2 What Does Big Data Mean for Information Systems?................. 169
8.3 Data Storage and Database Management Systems...................... 170
8.3.1 Database Management Systems.................................... 172
8.3.2 Key-Value Databases .................................................... 173
8.3.3 Online Transactional Processing (OLTP) ..................... 173
8.3.4 Decision Support Systems (DSS) ................................. 175
8.3.5 Column-Based Databases ............................................. 176
8.3.6 In Memory Systems...................................................... 176
8.4 What a DBA Worries About......................................................... 177
8.4.1 Scalability ..................................................................... 177
8.4.2 Performance .................................................................. 178
8.4.3 Availability.................................................................... 179
8.4.4 Data Migration.............................................................. 179
8.4.5 Not All Systems Are Data Intensive ............................. 182
8.4.6 And There Is More to Data than Storage ...................... 183
8.5 Open Source................................................................................. 183
8.6 Application Packages................................................................... 184
8.6.1 Open Source vs Vendor Supplied?................................ 186
8.7 The Cloud and Big Data............................................................... 187
8.8 Hadoop and NoSQL..................................................................... 189
8.9 Summary ...................................................................................... 190
8.10 Review Questions......................................................................... 190
8.11 Group Work Research Activities.................................................. 191
8.11.1 Discussion Topic 1 ........................................................ 191
8.11.2 Discussion Topic 2 ........................................................ 191
References................................................................................................. 191
9 Sources..................................................................................................... 193
9.1 What the Reader Will Learn......................................................... 193
9.2 Data Sources for Data – Both Big and Small............................... 193
9.3 The Four Vs – Understanding What Makes Data Big Data ......... 194
9.4 Categories of Data........................................................................ 196
9.4.1 Classification by Purpose.............................................. 196
9.4.2 Data Type Classification and Serialisation
Alternatives................................................................... 198
9.5 Data Quality ................................................................................. 202
9.5.1 Extract, Transform and Load (ETL) ............................. 205
9.6 Meta Data..................................................................................... 206
9.6.1 Internet of Things (IoT) ................................................ 206
9.7 Data Ownership............................................................................ 209
9.8 Crowdsourcing ............................................................................. 210
9.9 Summary ...................................................................................... 212
9.10 Review Questions......................................................................... 212
9.11 Group Work Research Activities.................................................. 212
9.11.1 Discussion Topic 1 ........................................................ 213
9.11.2 Discussion Topic 2 ........................................................ 213
References................................................................................................. 213
10 IS Security................................................................................................ 215
10.1 What the Reader Will Learn......................................................... 215
10.2 What This Chapter Could Contain but Doesn’t ........................... 215
10.3 Understanding the Risks .............................................................. 216
10.3.1 What Is the Scale of the Problem?................................ 216
10.4 Privacy, Ethics and Governance ................................................... 218
10.4.1 The Ethical Dimension of Security............................... 218
10.4.2 Data Protection.............................................................. 221
10.5 Securing Systems......................................................................... 224
10.5.1 Hacking......................................................................... 224
10.5.2 Denial of Service........................................................... 225
10.5.3 Denial of Service Defence Mechanisms....................... 226
10.5.4 Viruses and Worms and Trojan Horses
(Often Collectively Referred to as Malware)................ 227
10.5.5 Spyware......................................................................... 228
10.5.6 Defences Against Malicious Attacks ............................ 228
10.6 Securing Data............................................................................... 230
10.6.1 Application Access Control .......................................... 233
10.6.2 Physical Security........................................................... 234
10.6.3 Malicious Insiders and Careless Employees................. 235
10.7 Does Big Data Make for More Vulnerability? ............................. 235
10.8 Summary ...................................................................................... 235
10.9 Review Questions......................................................................... 236
10.10 Group Work Research Activities.................................................. 236
10.10.1 Discussion Topic 1 ........................................................ 236
10.10.2 Discussion Topic 2 ........................................................ 236
References................................................................................................. 237
11 Technical Insights.................................................................................... 239
11.1 What the Reader Will Learn......................................................... 239
11.2 What You Will Need for This Chapter ......................................... 239
11.3 Hands-on with Hadoop ................................................................ 240
11.3.1 The Sandbox ................................................................. 240
11.3.2 Hive............................................................................... 248
11.3.3 Pig ................................................................................. 251
11.3.4 Sharing Your Data with the Outside World................... 256
11.3.5 Visualization ................................................................. 258
11.3.6 Life Is Usually More Complicated!.............................. 261
11.4 Hadoop Is Not the Only NoSQL Game in Town!........................ 262
11.5 Summary ...................................................................................... 263
11.6 Review Questions......................................................................... 263
11.7 Extending the Tutorial Activities.................................................. 263
11.7.1 Extra Question 1 ........................................................... 263
11.7.2 Extra Question 2 ........................................................... 264
11.7.3 Extra Question 3 ........................................................... 264
11.8 Hints for the Extra Questions....................................................... 264
11.9 Answer for Extra Questions......................................................... 264
11.9.1 Question 1 ..................................................................... 264
11.9.2 Question 2 ..................................................................... 265
11.9.3 Question 3 ..................................................................... 265
Reference .................................................................................................. 266
12 The Future of IS in the Era of Big Data................................................ 267
12.1 What the Reader Will Learn......................................................... 267
12.2 The Difficulty of Future Gazing with IT...................................... 267
12.3 The Doubts................................................................................... 268
12.4 The Future of Information Systems (IS)...................................... 269
12.4.1 Making Decisions About Technology........................... 271
12.5 So What Will Happen in the Future? ........................................... 275
12.5.1 The Future for Big Data................................................ 275
12.5.2 Ethics of Big Data......................................................... 279
12.5.3 Big Data and Business Intelligence .............................. 280
12.5.4 The Future for Data Scientists ...................................... 281
12.5.5 The Future for IS Management..................................... 282
12.5.6 Keeping Your Eye on the Game .................................... 285
12.6 Summary ...................................................................................... 286
12.7 Review Questions......................................................................... 286
12.8 Group Work Research Activities.................................................. 287
12.8.1 Discussion Topic 1 ........................................................ 287
12.8.2 Discussion Topic 2 ........................................................ 287
References................................................................................................. 287
Index................................................................................................................. 289
http://www.springer.com/us/book/9783319135021
Big Data System Development
2015 Presentation by Chen et al.
https://sse.uni-due.de/bigdse15/BIGDSE2015-Chen_et_al.pdf
Addressing the Software Engineering Challenges of Big Data
POSTED ON OCTOBER 21, 2013 BY IAN GORTON IN ARCHITECTURE
https://insights.sei.cmu.edu/sei_blog/2013/10/addressing-the-software-engineering-challenges-of-big-data.html
Article - McKinsey Quarterly October 2011
Are you ready for the era of ‘big data’?
By Brad Brown, Michael Chui, and James Manyika
http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/are-you-ready-for-the-era-of-big-data
