Dominik Ryzko - Modern Big Data Architectures

The Chief Information Officer's Body of Knowledge: People, Process, and Technology by Dean Lane

Cloud Computing and Electronic Discovery by James P. Martin and Harry Cendrowski

Confessions of a Successful CIO: How the Best CIOs Tackle Their Toughest Business Challenges by Dan Roberts and Brian Watson

CIO Best Practices: Enabling Strategic Value with Information Technology (Second Edition) by Joe Stenzel, Randy Betancourt, Gary Cokins, Alyssa Farrell, Bill Flemming, Michael H. Hugos, Jonathan Hujsak, and Karl Schubert

The CIO Playbook: Strategies and Best Practices for IT Leaders to Deliver Value by Nicholas R. Colisto

The Complete Software Project Manager: Mastering Technology from Planning to Launch and Beyond by Anna P. Murray

Decoding the IT Value Problem: An Executive Guide for Achieving Optimal ROI on Critical IT Investments by Gregory J. Fell

Enterprise Performance Management Done Right: An Operating System for Your Organization by Ron Dimon

Information Governance: Concepts, Strategies and Best Practices, Second Edition by Robert F. Smallwood

IT Leadership Manual: Roadmap to Becoming a Trusted Business Partner by Alan R. Guibord

Leading the Epic Revolution: How CIOs Drive Innovation and Create Value Across the Enterprise by Hunter Muller

Managing Electronic Records: Methods, Best Practices, and Technologies by Robert F. Smallwood

On Top of the Cloud: How CIOs Leverage New Technologies to Drive Change and Build Value Across the Enterprise by Hunter Muller

Straight to the Top: CIO Leadership in a Mobile, Social, and Cloud-based World, Second Edition by Gregory S. Smith

Strategic IT: Best Practices for Managers and Executives, Second Edition by Arthur M. Langer and Lyle Yorks

Trust and Partnership: Strategic IT Management for Turbulent Times by Robert Benson

Transforming IT Culture: How to Use Social Intelligence, Human Factors, and Collaboration to Create an IT Department That Outperforms by Frank Wander

Unleashing the Power of IT: Bringing People, Business, and Technology Together, Second Edition by Dan Roberts

The U.S. Technology Skills Gap: What Every Technology Executive Must Know to Save America's Future by Gary J. Beach

Dominik Ryżko

© 2020 by John Wiley & Sons, Inc.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the Web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993, or fax (317) 572-4002.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Cataloging-in-Publication Data is Available:

ISBN 978-1-119-59784-1 (hardback)

ISBN 978-1-119-59794-0 (ePDF)

ISBN 978-1-119-59793-3 (ePub)

Cover Design: Wiley

Cover Image: © Pobytov/Getty Images

Over several years of my career in IT, I have observed how various ideas and technologies have come and gone, taking different paths, from being new and innovative to maturity and adoption, only to be replaced by even newer concepts as they arrive. Some gained popularity very quickly and became the buzzwords of their time, something that everybody tries and claims to be doing. Such is the case of big data , the popularity of which skyrocketed and was embraced by research, industry, and governments alike. In 2012 the Obama Administration announced the Big Data Research and Development Initiative [2012] acknowledging it as a key enabler to accelerate the pace of discovery in science and engineering, strengthen our national security, and transform teaching and learning . Only recently has big data been overshadowed by the widespread adoption of artificial intelligence (AI), which by the way, builds on the foundations of big data. However, big data will remain strong for the foreseeable future.

Other promising technologies from the past have not stood the test of time. The rise of statistical approaches to AI, and deep learning in particular in the past decade, gave the final blow to the symbolic methods, which I found elegant and fascinated me at the time of my undergraduate studies in the 1990s. Have the logical systems passed forever? Possibly not; after all, there are still open questions as on how humans analyze facts, reason, and make decisions, which we are not yet able to model purely by statistical methods. Only the future will show us in which direction science will progress.

Another interesting story is related to Multi-Agent Systems (MAS), the field I picked for my PhD and later research. While the concept of an agent, or closely related actor, go way back to the 1970s, it never gained wide popularity outside the relatively narrow research community and some niche business applications. Despite bringing in innovative views on information system paradigms and promises of solving some of its challenges, the idea never became widely adopted.

As my focus started to switch toward industry projects and I became more and more involved with building large scale big data and AI systems, I observed that some of the fundamental assumptions behind MAS have made their way into mainstream information systems. Monolith systems, dominant at the beginning of my career, started giving way to services and later to micro-services. Physical devices distributed in the physical environment gained computational power and thus built-in intelligence and increased autonomy. These changes were happening not because MAS were gaining in popularity, but due to the fact that it made sense to solve real issues in this way.