CS3
I made up that name, as well as its predecessor, which was actually a better name. That first name was SCADAnet, but I had to come up with something else that was catchy for Boeing IT political reasons.
CS3 had a nice techy ring to it without actually describing what it was about, or getting the Boeing network folks too much up on one of their high horses of defending what they saw as their rightful turf. It was also an attempt to teach them a bit about what I came to call The Tao of IT by simply demonstrating the ideas in action. It was an attempt to teach by an example.
CS3 is short for Control System Security Solution, and this was our response to that problem I described in the story about Mike Kozak and his phone call.
Before explaining the objective of CS3 and the solution to the 777 moving line network problem, let's look at the history of networking personal computers. In 1983 when I first started working on networked PC's, adding one computer to a network was an all day affair, if we were lucky. If we had to buy stuff, it could take weeks. By 1987 we had that down to about an hour using Novell Netware and saturated station wiring (a wire pulled to every desk in an office bay with a data closet containing enough electronic equipment capacity to make any station "live" as needed. When we started rolling out WiFi in 1997 we got that down to a few minutes. By the mid-2000s we were totally hands-off, with users connecting their own PC's using WiFi whenever and wherever they happened to find it convenient to be connected. It was a nice process of continuous service improvement.
Unfortunately, when it comes to things like providing secure network connectivity, it often takes as much time to day in 2022 as I write this as it did back in the early 1980s. Our challenge for 777 was to make the networking lives as easy for network control system engineers in a Boeing factory as easy as it was for the average laptop user, but with the security that such systems demand.
Factories are dangerous places. One has to be very safety conscious when in one. That means having an awareness of the local hazards and how to spot them so as to stay out of harm's way. Also, the computerized control systems of all of the equipment has to be protected. The potential for a nefarious actor to hack a system has to be taken to zero. And, the American aerospace industry and its factories are prime targets, so the level of security required is dramatically higher than that for a laptop. We're up against well funded state actors who are highly motivated to do mischief in our facilities.
The perfect network security solution is invisibility. We have a saying in this business: "If you can't see it, you can't hack it."
Unfortunately, the way standard networking is done using internet protocols, in most cases, security is something that is added on top of layer 3. There are standard tools for doing it at layer 2, but those tools have not changed since they were first made commercially available in 1991. And, as one might expect, they were designed for the exclusive use of the network support team, and not exposed to the users. In other words, the control systems people would need to submit a service request, often with budget support, to the IT organization for every last tiny adjustment to their systems. It is a total nonstarter of a disastrous IT customer non-service situation.
In his seminal book The Psychology of Everyday Things (POET), Donald Norman made the observation: "When everyday things require instructions it is a sure sign of poor design." In the world of consumer products, the kinds of stuff one expects to buy at a store like Target, if a customer goes looking for a customer support phone number in the instructions of packaging, the product is considered to be a design failure. If customer support is required, there is an obvious opportunity for product improvement. It's a terrible product design outcome when a customer believes that a product of service should meet this consumer standard, and becomes angry because they have to ask for help with something that they expect to be easy for themselves to do. As bad as that situation is, it is even worse when an organization is deliberately defending their turf and forcing customers to needlessly ask for help. And yet, that is exactly the situation with control systems networking in most situations.
Let's look at two areas of network services: Personal Devices such as a PC or smartphone, and control systems. Specialty boards to allow early Apple ][ and the original IBM PC to be used as dumb terminals for mainframes and other large computers were available as early as 1981, and perhaps earlier than that with the Apple machines. Local area network boards like the 3Com 3C501 Ethernet board for the PC were available by the end of 1981. So let's start there.
In 1981 if you wanted to connect your IBM PC to an Ethernet, it required a highly skilled computer systems engineer to do it, and it would take them several hours. Basically, you had to have a friend. There was no such thing as submitting a request to an IT organization to get it done.
By 1991, WiFi had not been invented, and PC-LAN systems had not yet converted to TCP-IP, so if the place you wanted your connection already had a wired station drop or cable available, it could be done in a few minutes, once the specialized technician got to it. Typical turn around times for an IT department doing this were on the order of several days to a week or more.
IBy 2001 with WiFi starting to be more common, a knowledgeable user could do it themselves, but more typically it would require a call to a help desk and have them walk the average user through it. Fifteen minutes was common.
By 2011, it was expected that all PC users could do this form themselves with no help or support of any kind being required. So it was basically instantaneous.
Black art to users doing things themselves as needed was accomplished in 30 years of steady progress. To be sure, there are still some massive issues with privacy and security with things as they stand for the individual personal device user, but progress at making their task of getting connected when and where they wanted has been steady. Alas, this has not been the case for industrial systems engineers and their control systems.
This is where you are!
For the benefit of those who simply must know right now what that hidden complexity is about, it is something called "ephemeral ports" which are normally hidden from view, even when we see a super long and fantastically complicated web address. If the true full address were shown, those long addresses that take up three and four lines of text would require something more like six or seven lines. A quick hint at what ephemeral ports are about can be seen when you open several browser tabs to the same location. A web address looks the same in each tab, but obviously it can't be. Something has to direct inbound data to the right tab. That something is an ephemeral port assignment, and in any one comlink there can be a great many hidden bits of ephemeral port addressing shorthand.
Layer 4 is the Transport Layer. This is where we setup the rules for how to add data to packets, which is the stuff we care about. Think of it as the rules the post office has for what sort of envelopes and boxes they want you to use for various things, and how to properly seal them to survive the trip. In the case of your network packets, there are also some rules that allow a bunch of data to be broken up into multiple packets and later properly reassembled at their destination.
Let's skip layers 5 and 6 for now. We're rattled on way too much as it is. Let's do mention Layer 7, but only a little. This is the Application Layer. It is where things like your web browser can do things that are unique to using the web, or your email client can do things that are only about email, most streaming media apps can do their special thing, and so on.
With that we have enough of the basics to talk about the projects I worked on in the Boeing IT central architecture organization.