Friday, 31 October 2014

The day the country came to a standstill (the need for end to end change management)

One morning I was driving in to work and was listening to a CD and switched to Radio 702 just before 7. The headline news was about major delays at Johannesburg International Airport (now called OR Tambo) that was caused by a computer problem.
Read about what happened here.

Madge Micro Channel Adapter

Madge CAU with STP LAM

Madge Network Adapters

The Old Ten pack!
EISA adapter.

Sick! The hidden message from American Chopper for technologists

American Chopper on Discovery is one of my favourite TV shows. I often have sat on the couch and wondered why I was so fascinated by the Teutuls. I left it at, enjoyment. I also suspected that the pressures of working against deadlines and the bombastic and aggressive behaviour hit an accord with my work in Information technology (IT). However, I revisited the theme after attending a presentation by Peter Armstrong, where he asked the question of what was the purpose of a rev counter in a car. He assets it serves no purpose to the underlying functionality and purpose of a vehicle or is of any use to the driver. Now I wouldn't buy a car without a rev counter, so if Peter's statements are correct, what is the underlying reason for that damn rev counter. There is no enjoyment for me in a car without a rev counter. I once drove an old sports car without one, but the loud sound of the engine in that case was a more than suitable substitute. I eventually found the answer in American Chopper.

American Chopper is about making motorcycles that are sick! Sick means cool, good looking and aesthetically pleasing (heck, I can feel Paul Sr warming up his knuckles for that last description). The bottom line is that the Teutils won't make a chopper unless it look good. Yes, it must be functional and have a purpose but it must be sick!
Now most software and hardware that is delivered by technology companies is nauseating. I suggest you reply as a comment with your suggestions of the worst examples in both hardware and software. These products look the totally opposite of sick! The assertion is that it does not need to look good if it is functional. Well, I think they have it all wrong!

Firstly, the aspect of quality is overlooked. A company that has been detailed enough to spend time on designs that make it look good, would have spent greater due diligence on the functionality. A better looking product will be technologically better.

Secondly, there is the comfort. Product sets with rev counters, although delivering no functionality benefit, serves the purpose of harmonizing and making the purchaser of the product comfortable. This results in a satisfied customer, and a satisfied customer is the best advert for your business. As an example, I have missed the LCD panel with the paddle switches on Madge token-ring CAUs and switches. That was sick. The current generation of Ethernet switches with dull colours, bad designs and stupid flashing Christmas lights don't move me to confidence. It comes down to that fundamental genetic instinct to kick the tyres of a car. Admit it, you do it although there is no functional purpose to do it.

Before, I am cast out as a heretic into the wilderness, let me offer an example in Apple. Apple customers are the most vehement supporters of their products and I believe that the design of the products is core to this characteristic.
The hidden message to technologists is that don't stop when you have a functional product that satisfies a business requirement. Go the extra mile, to make it sick.

Token-ring memories

This is guaranteed to bring a tear to Simon's eye as he still can't bear the thought of putting his old token-ring equipment in the skip. Many moons ago through the Cape Town and Atlanta airports and at both locations I noticed token-ring networks. Please click on the comments field below this blog and tell me where else you have see token-ring still refusing to die.
While searching for a CD, I discovered a CD copy of my old Thinkpad with some video clips on it. So I created a YouTube account and uploaded them! I hope these memories will delight all the members of the old Madge. To view the HSTR seminar presentations click here and here.

Robert Madge on support without limits.
Madge Communications.
Madge Plant Tour.
HSTR Launch - Mike Wilson
HSTR launch - Jeff King and Simon Gawne.

HSTR LAB demo with Mike Cohen.

Sharon Wilber on HSTR alliance.

Jeff King doing a HSTR demo.

Madge South Africa

Go Karting in Spain.
On holiday in Ireland.
Argus Cycle Tour

The Legend...

The Madge Token-ring adapter, 16/4 BM2, the legend! Best network adapter ever shipped to market!

The best adapter as voted for by the ex-Madgers at Nortel.

The Madge AT Plus - damn, I had to search long and hard for this picture.

Kate Godfrey: "Myself and my ex Madge colleagues at Nortel have decided that the best adapater was the AT+ (I was voted down on the FDDI card) and the worst was the Smart16........ so we don't concurr with the link :-) BTW: Whatever happened to Mr. R. Madge,,,,,,,, that should be on your blog."

My visit to Duncan at 7DaysTech

Today I had a cup of coffee with Duncan of 7Daystech. We had a chat about Vontu and then took I a picture above of the plaque of the Madge Africa opening.

Network visualizations

Many moons ago on the Netcordia blog, I read an interesting blog by Terry about useful visualizations. This triggered my thoughts about network visualization.
There are two distinct types of visualizations and I'll provide my opinion about both, i.e. real-time and static. Unluckily, there is no network management vendor who really provides a decent visualization. In reality the best I have seen was dated pre-1995, written as a DOS application by Madge Networks. It provided real-time visualization of source routed spanning tree networks. It was a great tool in troubleshooting problems in token-ring networks, and displayed bubble, stick and lollipop diagrams. (Damned if I can remember the name of the app!)

I have specified two types of visualizations as each serves as different functional requirement. The real-time visualization is useful when the !@#$%^ has hit the fan, and the static visualization is useful to prevent the !@#$%^ from hitting the fan.

Real-time visualizations
The best visualization is RADAR. This type of visualization has been around since WWII, but hasn't found it's way into a network management product. The basic idea is there, see this post about the history of ping, but a product to fully emulate radar in networks has not been done.
Radar as used in air traffic control provides height, speed, direction and location to a controller. The controller also creates on his scope a depth by stipulating the radius being monitored. Importantly, radar does not monitor all aircraft. It sequentially scans the skies.
How does a network radar look:
  • the controller determines the depth by stipulating the maximum latency to a network device that will appear on the scope.
  • the height of the network devices are the aggregated packet / byte count for the designated monitoring period.
  • the speed of the network device is the packet / byte rate at the time of the poll.
  • the location is the IP address which also determines the radius on which the network device appears.
  • the direction of the network device is the direction of the single biggest flow, with the direction being determined by the IP address.
Static visualizations
The best static visualizations are street maps. The routers are the roads, the links are the buildings, the campuses are the office parks, and the traffic load determined the height of the building.
Routers that are directly connected to each other are represented as cross roads.
How does a network street map look like:
  • the width of the road is determined by the router's processing ability.
  • the size of the building is determined by the link's speed.
  • the height of the building is determined by historic traffic load.
  • colour is used to designate topology types.

There is a related blog post, taking the theme further using auralization (visualizations using sound.)

What testing does not capture! (time lapse)

Many moons ago I was reviewing an interesting Field notice from Cisco. A MDS switch reboots after 233 days of operation. Now imagine the surprise of the IT guys when it happens. "Jack! Did you trip over the power cord, again!" "Boss, I swear, I did nothing, I was having a smoke outside!"
Now testing never captures a time lapsed fault until it happens! The most appropriate strategy is to have redundant systems and updated them sequentially. This may sound logical, but once I encountered techies updating (patching) a redundant system at the same time. When I inquired as to why this obviously flawed strategy was being used, I was informed that this is what had been decided by the change forum. The reasoning was that since there was going to be downtime, they wanted to minimize the downtime. The strategy might have been correct for a time analysis of the task but the risk of time lapsed bugs had been discounted. The resultant downtime as a result of a major incident makes it crucial that updates are planned in a sequential manner.
If the update results in the redundant systems not being able to maintain its state, because they are on different levels, due to version incompatibility, then besides kicking your vendor's backside to kingdom come for bad practice then it is better to have the system function with limited (or no) redundancy for a short time period. This strategy has less risk associated with it, than a simultaneous time lapsed issue causing a major incident on your redundant systems.
Imagine the surprise in the data centre operations area when a failure like this occurs. "Don't worry mate. No biggie. We have two of those. What the heck! The second one is also rebooting! RED ALERT! SEV 1! WAKE THE NEIGHBOURHOOD UP!"
Years ago I remembered a similar type of issue with the Madge SmartCau Plus, a token-ring network hub. The code had a bug that caused the hub to lock-up after 255 days. The developer had used a variable for recording the number of days of up time and had defined it as a byte data type. However, the program logically used the variable as a word data type and at day 256 the variable overflowed, and KAPOW!
When we had realised the cause of the lock-ups, we immediately had customers do a controlled reboot as a workaround, after which we had 255 days in which to load the new code without the bug.

Lesson learned: Patch sequentially.
Reality: Super Tuesday habits. Patch everything at once.
Warning: There will always be major incident tsunamis, due to a time lapsed bug

Where in the world is Robert Madge?

Many people emailed me to ask what is Robert Madge doing these days...

Robert Madge is the President of IDtrack (the European Association for Secure Identification), a non-profit organisation based in Barcelona, Spain, which provides information, guidance and coordination for the secure implementation of radio-frequency identification (RFID) and other techniques of automatic identification.Mr. Madge is also the founder of Xifrat Daten AG, of Zug in Switzerland, which develops software technology for large scale database implementations, and which produces solutions for process flow and for data-intensive product tracking and tracing applications (arising from the use of RFID and other data gathering technologies).
In addition, Mr. Madge is the founder and International Director of FoodReg AG, which provides automated record-keeping and controls for the food industry and which is a pioneer in food traceability solutions and computerised food safety programmes. Furthermore he is the founder and Chairman of Olzet Solutions SA, which provides consultancy and project implementation services across Europe to companies implementing product traceability and supply chain execution projects, and the founder of Olzet Seguridad Alimentaria SA, which provides similar services to the food industry in Spain.
From 1986 to 2001, Mr. Madge was the Chairman and CEO of Madge Networks NV, a company in the field of networking technology and products. Under Mr. Madge’s leadership, Madge Networks grew to become one of the world’s leading network product companies in the mid-1990s, producing innovative products based on the technologies of token ring, ATM, multi-media, and wireless networking. The company’s operations extended to offices in more than 40 countries, and annual sales of $500 million.
Prior to 1986, Mr. Madge was the Technical Director of Intelligent Software Ltd, an R&D company in the UK, and in 1981 was the inventor of the world’s first electronic organiser (which were the forerunners of PDAs).

Why there will always be a higher rate of major incidents on Cisco based networks

There are two underlying fundamental problems in Cisco based networks that will always cause them to be the cause of a higher rate of major incidents compared to alternative vendors. The problems are related to the high number of features that the vendor shovels into it's code. The higher the count of features, especially those that are unused, the higher the potential for faults. Additionally, unlike JUNOS, IOS does not have a single linear code versioning methodology. This results in deployment and configuration issues. Nearly, 14 years ago, I learnt the benefits of a single linear code versioning methodology from Madge Networks adapter software, called LAN Support Software (LSS). Although, there were ISA, MCA, PCI and EISA adapters they were all supported by the same single version of LAN Support Software. This had a great impact on reducing the rate of major incidents, a lesson which Cisco still needs to learn. These are the benefits of LSS in which IOS has no related equivalence:
  • Largest range of Operating Systems supported in a single release.
  • Single, high performance, ‘UniDrivers’ for PCI and PC Card adapters. This means that for most operating systems, a single driver supports Madge Token-Ring adapters.
  • Microsoft ‘signed & certified’ drivers for Windows XP, 2000, Windows ME and NT4
  • Enhanced server support: NetWare, PCI Hot Plug, Adapter Mirroring
  • Support for legacy OS/2 & DOS NDIS environments
  • Support for Linux, Solaris, HP-UX and MAC OS
  • Upgraded and Updated installation and configuration utilities
  • Supports advanced features of Madge adapters, such as ACPI power management and Wake-On-LAN.
  • Adapter mirroring for Windows and NetWare increasing server resilience and fault tolerance.
Yes, it is token-ring and token-ring is dead, but the lessons learnt are definitely not! In his blog, Tony Rybczynski, of Nortel, writes in a fuller and detailed substantiation of these views:
"Unfortunately, data networkers buy technology based on the Cisco feature list (who have hundreds of features that seldom are used by any given customer) rather than on their feature requirements.
As a result, they pay premium prices, and create unnecessary complexity which can impact performance, security, reliability and TCO.
On reliability alone, it’s no secret that software complexity (multiple versions of IOS, features you don’t use, and resulting config errors) is a major contributor to failures."

The Madge CAU and LAM launch in the United States

My mate Dave Ruby emailed me the above picture of the launch in the United States of the Madge CAU and LAM. The CAU and LAM operated as a token-ring hub. Dave is the guy in the middle becoming better acquainted with the cow. The Madge CAU was a solid bit of kit that rarely went titsup, and compared to the IBM CAU 8230, it was the bull's bollocks. When it came to IBM, we never turned the udder cheek!
Eventually, Madge went titsup and IBM ratted out to Cisco.

Auralization - short wave radio for network management

Many moons ago I was reading Terry’s blog when he was at Netcordia, where he posted about auralization which reminded me about my own experiments in using voice for monitoring a network. Not strictly auralization, but stay with me. A few years ago, I created a monitoring system based on Suse and using Argus. I created a call me system using text to speech and Openh323. The voice integration was driven using an old Radvision H.323 gateway that Madge OEM'ed way back in the 90s.

What this monitoring system did was phone me at a designated number and tell me in an automated voice message what the problem was. No SMS or pager bull, just a plain voice (I could never read a SMS at 2:00am!). I also had a feedback channel via DTMF. It cost zilch and worked like a charm! The gateway was an inheritance from when Madge went titsup, the server was an old Internet caching appliance and the software was open source. My code is long lost, but the strength of using audio and voice has remained in my thoughts. Is there any software out there today that does the same thing? I also thought of doing a flight director poll. The idea would be that I call in and the system would give feedback in much the same way the flight director of an Apollo mission would have heard in Mission Control. “Backbone we are go”, Kalahari Edge, we are no go as the systems are at 90%.”, "Backup we are go." etc. Never got around to doing it as I became a manager and lost all of my ability to program. :-)
Ok, after that slight diversion back to back to Terry’s post about auralization which is the same as visualization but with sound. My idea would be the short wave radio. The short wave radio has many possible auralizations:
  • Hum variation based on network utilization.
  • Static pops based on errors
  • Feedback loop spikes based on failures
  • Morse code alerting SOS . . . - - - . . . for a major incident as an example
  • Automated tune in to weather report or traffic report, e.g. "email from the Kalahari to the outback has increased resulting in a jet steam congestion".
  • Random timed tune in to the flight director poll described above.
  • Tornado warnings based on incidents spikes at the service desk.
  • or basically any other radio-centric voice or sound effect.
There are many sources for determining sounds, including syslogs, snmp traps, netflow, icmp and ipsla. In a large network the amount of information being generated by the network and the number of issues can be relatively high and the sounds can become a trigger for proactively detecting major incidents.
However, the big thing is, does the younger generation recall short wave radio???????? I have an old 1948 Murphy, so maybe the above just appeals to me??

Living the values

Most companies do not realize that the priority in life for individuals lies elsewhere.

Read about my views here.

Support without limits

Madge Networks had a simple philosphy of "support without limits." It meant that if you bought a Madge product you could be guaranteed that the company would assist you unconditionally in the use of that product and not charge you extra. They did go titsup, but that is a different story.

View the video here where Madge explains about the philosophy.

Fourteen years, and many service management conferences, talks and workshops later, I still haven't witnessed an enterprise or vendor deliver to the same service quality as Madge.

Many companies and individuals are selfish and instead of focusing on adding value to the people that they are assisting, they instead rate and concentrate on their own worth. It is about, "what is in it for Moi?" Instead the primary focus should be about what Richard Branson states, "about doing the right thing," as blogged about here.

Ye olde Madge Networks datasheets

Supplied by the famous Mister Stubbs!

Obituary: The Cow and lamb are dead

Over at Denver, the IEEE finally closed the book on 802.5, a.k.a. token-ring. The standards have been withdrawn. Token-ring is well and truly dead.
However, over at the Thinking problem management blog, there is a section dedicated to token-ring memories. If you have any Madge or token-ring memories and pictures email them to me at ronaldxbartels at and I'll post them. Here is an example: The Madge Networks CAU and LAM launch in the US!

News articles about Robert Madge

The Economist, dated December 1995, "Madgical mystery tour"
The International edition of Business Week April 1995 "Entrepreneurs - Can they thrive in Europe" Robert Madge is on the cover.

The London Sunday Times "Britain's Richest 500 1994" Robert Madge was 55th that year.

John Detwiler's Mugs

The blue one says "To commemorate the visit of Her Majesty The Queen to Madge Networks 18th November 1994". She presented an export award to Madge.

Madge LANNET LET36 chassis

Madge LANNET LET36 MULTINET II Chassis. I spend some time with this chassis in various locations. It was better and more stable than the Visage that followed!

The fall of Madge (Tennant's podcast on Madge)

"Does the name Robert Madge mean anything to you? It probably rings a bell, if a distant one. Madge was the founder of Madge Networks, which in the late ‘90s was the market leader in Token Ring networking technology. The once high-flying company has since met the same fate as that of Token Ring itself: near oblivion.
The story of the collapse of Madge Networks is a remarkable one, especially when you hear it directly from Madge himself. That's what happened over breakfast earlier this week when I met up with Madge at the 16th World Congress on Information Technology in Kuala Lumpur, Malaysia. Madge, who since leaving his namesake company in 2001 has won acclaim in the field of RFID and tracking technology, was completely open with me about the downfall"

The above podcast of Computerworld's Tennant is about the fall of Madge.

The Queen visits Madge Networks at Loudwater, November 1994

Do you recognize the people in these photos, besides the Queen and Robert Madge? Leave a comment on the blog if you know!

The Madge Smart CAU

A trip down memory lane, when men were men and networks were token-ring.

Madge Smart Ringswitch Version 3

The Ringswitch, "Cadillac of the network!"

Madge Smart Ringswitch Version 2

The version that went dirty with transparent switching!

The original Madge Networks engineering team

The original engineering team at Madge Networks: Bruce Tanner, Mark Richer, Martin Lea, Robert Madge and Rob Stubbs.

High speed token-ring

Although Madge Networks has faded from memory, and exists really only as a Wikipedia enty, there are still items on ebay! Like the HSTR blade from the Ringswitch.

If there are still live installations, send in your pics to ronald.bartels at!

Stewart Marshall's Madge memories

Stewart Marshall emailed me these Madge photos from his archive.