Categories
Industrial IT

On Openness

I’m Jason Firth.

Merry Christmas, and happy new year!

On my “About me” page, I wrote: “With this blog, I have a few goals: I’m hoping to get some of that information together so control professionals from all over can use it. I’m hoping to take some of the extremely cryptic academic work out there and simplify it for industry.”

Recently, I was speaking with someone from the aaOpenSource project, which was started in part by the guy at the Archestranaut blog over at Avid Solutions. I definitely recommend the blog. It isn’t always updated, but when it is, there’s some great information there.

One thing we both agreed on was that this industry needs more openness and sharing.

I started my “programming career” such as it is in open source. I started off by learning GWBasic, then progressed to QBASIC, then learned Visual Basic and C++ and a bunch of other programming languages afterwards. It might be a bit sacreligious for the hardcore programmers out there, but I always enjoyed BASIC, because compared to many other programming environments, you don’t need to micromanage as much. The runtime library contains most things you’d need for simple programs, so you don’t need to manage library binaries or header files. Eventually, I ended up using the FreeBASIC project. It’s very much like a C++ compiler with a very comprehensive runtime library built in. I ended up contributing a small amount of code, and working as much as I could to improve the documentation for new users.

No matter what programming lanugage I was learning, whether it was gwbasic or C++ or assembly or php, open code was a crucial piece of my learning experience. It was much easier saying “What does correct code look like?”, than trying to decipher sometimes archaic documentation. Having a library of code snippets to call upon means you can focus on solving the novel parts of your solution, rather than reinventing the wheel.

Two pieces of code in particular were things I was particularly proud of improving upon when I was back in high school were a graphics routine, and a keyboard handler.

In DOS programming, and particularly real-mode DOS programing, you end up having to manually handle your graphics to a large degree. I found some code demonstrating a simple pixel set routine for 320×240, a video mode called “ModeX”. It has some really cool features, such as allowing you to draw to an off-screen part of the video memory while showing a different part of the video memory. This is called “Double buffering” when there is an onscreen page and an offscreen page, but ModeX supports two offscreen pages and one onscreen page, called “Triple Buffering”. The most difficult part of programming this to run quickly is that there’s all sorts of insanity in how you write pixels properly. You have four “planes” which you have to write to, and each plane has the graphic laid out in an odd way. The original code showed me how to initialize the video mode, but the code for placing a dot on the screen involved calculating the memory location (involving a multiply and a divide), and setting the plane. After months of staring at the code, I came up with a clever way of writing an entire plane’s pixels in one step consisting only of additions, then I could write an entire screen with only 4 plane shifts. Without the original code showing how ModeX worked, I would have had nothing to start from, and I probably wouldn’t have gone with the arcane video mode without some sample code to start from. Without open documentation, the person who wrote that code probably never would have had a place to start.

Another challenge is key detection. For multikey applications, you have to capture each button press and unpress to determine the keyboard state. To accomplish this, you must create an interrupt handler to replace the existing DOS keyhandler, which only captures one key at a time. Then, you must continuously poll the port. I found some novel tweaks to the code to allow more accurate polling of the port and recording (and retrieval of) multikey values. Without the original code showing how raw keyboard polling worked, I would have had nothing to start from, and I probably wouldn’t have gone with any sort of continuous multikey detection without some sample code to start from. Without open documentation, the person who wrote that code probably never would have had a place to start.

These are small programming problems, but they’re how I started to learn. Without the documentation and open code, I never would have had a place to start, and never would have learned the fundamentals I use to solve problems on a regular basis today, a decade later.

However, our industry is built upon certain open standards. The PID, for example, or Zeigler Nichols tuning, or 4-20mA, or 3-15 PSI. Everyone who learns about the trade needs to learn these things, and by learning them, doesn’t need to reinvent the wheel later.

One thing that should be immediately obvious is that all those standards are from 40 years ago. In some ways, it’s like our trade hit a time warp, and although we’re seeing more and more new technology, it’s all a black box. Some specialized experts understand them, but they’re in the minority.

I come from a few industries where people believe that if you hoard information, and ration it out in little bits, that’s how you stay valuable. I don’t believe that. I believe that the way we stay relevant is by proving to the world all the interesting ways we can provide value to their organizations. We’re tube benders, but we’re not just tube benders. We’re cable pullers, but we’re not just cable pullers. We’re calibrators, but we’re not just calibrators. We’re documenters, but we’re not just documenters. We’re programmers, but we’re not just programmers. We’re electronics techs, but we’re not just electronics techs. I could go on all day, because our trade and profession is so broad, we end up with a view that is equally broad. Instead of being jealous and trying to protect this information, we should be teachers, trying to help each other, and also the other disciplines become better. If we try to go alone, to fend for ourselves, then we’re going to be swept away by the tide of all the new stuff we need to keep on top of.

That’s one big reason why I wanted to start writing this blog, because I was able to build upon the work of others, and I’d like to continue to do that. Together, we can build the future.

Thanks for reading!

Categories
Instrumentation

Siemens WS300 teardown

Siemens WS300 teardown

I’m Jason Firth.

I had an opportunity recently to tear apart a Siemens WS300.

I was hoping to troubleshoot a problem with it, and in the process I learned a bit about how they were put together.

Here you can see the mechanical components. The thick metal shaft is the part that sticks out of the sensor. It has a single bearing (the big round circular thing with the blue plastic) held in place by a retaining clip. Next, there was the circular piece of metal, and another bearing, which is then held in place with another retaining clip. The shaft with the bearings slides into the housing, and is held in place by the large retaining clip. The very end, where the shaft becomes small, has a little metal pin in it, which acts like a handle.

The orange part of the coupling has a slot cut in it, which slides over the shaft, holding onto it. Then, the orange part connects to the black coupling, which is held onto the binary encoder. The encoder is held to the circuit board with the metal clip, which screws onto the board, and holds the encoder with the large nut and locking washer.

The rotary encoder might look like a resistive potentiometer, but it is something a bit more complicated. Instead of changing from 0 to 100% resistance over the course of its range, it consists of a pair of lines which change differently in response to rotating in different directions.

Here’s the circuit board. Let’s look at some of the components:

There’s a LM317. This is an adjustable voltage regulator.

The AC74 looks like a dual flip flop chip (the model number isn’t exactly the same, so it’s possible it isn’t the right chip). This makes sense, because the output of this transmitter is a pair of signals: one changes when you turn the sensor in one direction, the other changes when you turn the sensor in the other direction. This chip is probably where our final logic comes from.

There are two transistors, Q1 and Q2. These probably take the logic signals from the AC74 and convert it into a full voltage signal.

The HC132A chip seems to be a quad NAND gate. You can create a lot of different logic using a few NAND chips, so this makes sense.

Knowing what these parts are, we now have a general idea how this works: The input voltage is regulated down, then some NAND gates and flip flops are combined to provide a pair of pulse outputs.

We determined that the rotary encoder was destroyed, since we now understood how to feed +5v signals to force the board to operate we were able to confirm that the electronics functioned and the sensor did not.

I hope you found this look at a common instrument as interesting as I did.

Thanks for reading!

Categories
Process Control

Wonderware System Platform 2014 R2 now available

I’m Jason Firth.

Yesterday, without much fanfare (at least for their customers), Schneider released Wonderware System Platform 2014 R2.

Some quick notes from reading the documentation:

  • Windows XP, all versions; Windows Vista, all versions; Windows Server 2003, all versions; Windows Server 2008, non-R2 versions are no longer supported.
  • Modern InTouch Applications now allow you to better integrate ArchestrA objects into Intouch.
  • Wonderware Intouch now incorporates the ArchestrA Graphic Toolbox directly.
  • Additional Situational Awareness Library Symbols – There are now a number of new symbols for alarms, valves, and trends.
  • Graphic Connectors – It seems a new tool is available that can connect different objects together.
  • Alarm Shelving – This seems like a particularly useful function that will let operators “shelve” a nuisance alarm for a certain period of time.
  • Enhancements to Programmatic Export and Import of ArchestrA Symbols – It looks like they’ve added new ways to create archestra symbols using xml.
  • Conversion of InTouch Windows to ArchestrA Symbols – This is a huge win for anyone who was going to be forced to convert their windows by hand previously. Anyone doing an Intouch to ArchestrA conversion knows what I’m talking about.

When I first saw System Platform for the first time, my instinct was that Wonderware Intouch was going to slowly fade by the wayside, and it seems this is happening. More and more, the much more powerful ArchestrA framework is being integrated more into Intouch.

I’ll use it more in the coming weeks, and write more about my experiences with the new system.

There have been a lot of steps in the right direction for Wonderware in the past few years: It seems like they’ve spent a lot of time resolving a lot of the show stopper bugs in the core program. I’m hoping to see that continue.

Wonderware customers with a support contract can find the new software on their website at The Wonderware Development Network.

Thanks for reading!

Categories
Trades

Women in trades and technology

I’m Jason Firth.

I’d like to suggest today, that women should look more at the trades and technology as a career choice.

Of course, there’s lots of people right now suggesting that. There’s people who are way smarter, and way richer, and way more powerful suggesting that. Obviously, another voice isn’t really that useful right now.

So I’d like to give the one thing I have that many of those people don’t: A view from the floor. I’m going to tell you why it makes a lot of sense to get into the trades and technology, I’m going to tell you about some of the challenges you can expect to face, and I’m going to tell you why the trades and technology is about a lot more than wires and ports.

Let’s start off by by looking at why the trades and technology are a great choice for anyone. Many jobs in the trades and technology are highly skilled, highly paid jobs that tend to have great benefit plans as well. Even tradesmen who lose their jobs, often find new ones way faster than others. Besides that, you get respect: If there’s one thing that people respect, it’s “I can’t do this thing, but this person can, and I need them.”

Besides that, the trades can be really fulfilling. Imagine, at the end of the day, looking at a building you’ve helped build, or light, or control. Imagine working in maintenance and being the person who solves the problem nobody else could, or getting a plant running that was down, costing tens of thousands of dollars an hour.

As for why women in particular should be interested, let’s start at the beginning of your career: When any woman I know says “I’m thinking of going back to school”, I immediately suggest taking an engineering technology course. As a starving student looking at scholarships and bursaries, it was immediately obvious that there’s thousands of dollars up for grabs in scholarships and bursaries for women in technology. There are plenty of other options for women, and many sort of “Traditionally female” choices out there, but why pay full price to follow what others do if you can get a helping hand to try something different? Besides that, taking an unconventional path has other benefits: Companies want to hire women for trades and technology positions. The problem is that the applicants aren’t there! The rewards are there for someone who wants to take the risk of doing something different.

Now, it isn’t always going to be easy.

Imagine being the first human on an alien planet, and the aliens don’t know how to be around a human. They might behave quite inappropriately at times. Unfortunately, because technology and the trades are so male-dominated, you get people who don’t know how to act with women in the workplace. I have to admit, you’ll sometimes be treated unfairly based on your gender. It isn’t acceptable, but it is real. It takes courage to lead, and the only way things will get better is for more women to engage.

Another thing that is true, is that some trades take more physical strength than others, and that can put women at a disadvantage. If you’re a millwright, you might be asked to really strain against a wrench. If you’re an electrician, you might be asked to pull really large cables. In these cases, you’re going to have to rely on your team to help, or find a different way to do the job. Not every job is like that. Instrument technicians don’t tend to need much brute strength for most of their jobs, and you don’t need arms like tree trunks to use a mouse and keyboard to draw in CAD.

But one criticism that I don’t believe is true is the image of trades and technology as a cold, inhuman field, all ports and wires and bolts, disconnected from people.

I’ve always been told that it isn’t just important that you can do these huge things, but that you can help others understand it too. Besides being a person who can do things, you need to be someone who can teach others, to help better understand what you can do for them.

Consider the largest company in the world right now by market cap: Apple. Steve Jobs wasn’t the first person to make a home computer, but his company made the home computer accessible for human beings. They weren’t the first to create a smart phone, but they were the first to make it accessible for human beings. They weren’t the first to make an MP3 player or on-line music store, but they were the first to make it accessible for human beings. It’s important to be good at your trade or the technology, but it’s equally important to be able to connect the technology with the people who aren’t you.

I personally spend a lot of time repairing and calibrating instruments, tuning loops, running cables, designing systems, and programming. However, I spend more and more time working with people who aren’t instrumentation and control specialists trying to help them understand what my team can do for them, or helping people learn exactly what tools we provide and how they can use those tools to help themselves, or describing to non-technical folks work we’ve done. The genius who doesn’t deal with people probably isn’t going to succeed compared to a person with less technical skill who is nonetheless willing to work with other people and other groups, and really help those groups make the best use of instrumentation, controls, and automation for the benefit of the organization.

There’s no reason why women should count themselves out of that.

Thanks for reading!

Categories
Instrumentation

Omega Engineering got smart

December 18, 2014

I’m Jason Firth.

We set up some Omega wireless temperature transmitters the other day, and I thought the method they used to get wireless connectivity is quite clever.

Take a look for yourself:

That’s an XBee Pro wireless transceiver.

XBee is a wireless communication standard for instrumentation that’s been made popular thanks to its association with Arduino microcontrollers. This popularity means that the chips are easy to acquire, and that debugging tools are simple to find and quite inexpensive.

Using an established protocol probably made their development costs way lower, and it also makes this whole system a lot more repairable. Really smart move, from where I’m standing.

Thanks for reading!

XBee Data Sheet

XBee video presentation

Categories
Process Control

Unity Pro 8.1 Released

I’m Jason Firth.

Because of an ongoing support issue we’ve been having, the folks at Schneider made sure to e-mail me about a point release of Schneider’s Unity Pro software that was recently released.

Additions include:

-support for new M580 devices

-Device Integration improvements

-references

-implicit conversions (no more real_to_int and int_to_real everywhere)

-security improvements

Interestingly, Unity Pro 8.1 supports Windows 8.1 but not Windows 8.0. Windows XP is no longer supported either.

It’s not mentioned anywhere, but there have been other bug fixes as well. I had a problem with animation tables when I built changes to the program, and installing Unity Pro 8.1 solved those problems.

We also had a problem where the program would slow to a crawl if the variable properties window (accessed by pressing ctrl-enter while a variable is selected) is open. This version solves that problem.

Thanks for reading!

Categories
Trades

The Path of the Certified Engineering Technologist for Instrumentation

I’m Jason Firth.

A while back, I wrote about some examples of situations where using unqualified instrument techs caused catastrophic damage and the potential for a massive loss of life. A few days ago, I wrote about one path to becoming a qualified instrument tech, the path of the Journeyman.

Today, I want to speak a little about another path a person can take to become qualified, the path of the Certified Engineering Technologist.

To understand what an engineering technologist is, we need to take a step back for a moment, and look at trades and engineering.

A tradesman (a journeyman, for example) learns the nitty gritty of making control systems happen: They learn the procedures for calibrating different instruments, the step by step instructions for accomplishing specific practical tasks. On the other end, you have the engineer, who intimately understands the theoretical frameworks that help them to develop new products. Between the two, you have an engineering technologist, who has substantial practical knowledge, as well as substantial understanding of theoretical frameworks.

An engineering technologist will have some of the tools of a tradesman, and some of the tools of an engineer. Where as tradesman will know how to tune a loop using Zeigler-Nichols, an engineering technologist will be able to tune the loop using Zeigler-Nichols, and will also be able to model a control loop in the laplace domain to determine stability, and can solve some of the simpler differential equations required. An engineer will be able to model a control loop in the laplace domain, but can also do a full thermodynamic, fluid dynamic, kinetic model to determine things far outside the scope of mere control systems.

Just like the path of the Journeyman, Certification of Engineering Technologists is handled by different legislation in each province. Because of this, the options open to engineering technologists can be quite different from province to province.

In any case, the beginning of this path is an accredited college. The program is called different things around the country. I went to Instrumentation Engineering Technology, but one program is called Computer control engineering technology, or automation engineering technology.

Regardless of the name of the program, you’re going to be looking at a lot of the same curriculum: Kinematics, fluid dynamics, electronics and electrical systems, calculus, linear process control, advanced process control, computer process control, final control elements, measurement, and quite a bit more. Along the way, you’ll have to complete a term project, and a written report.

Some programs try to cram the entire engineering technology curriculum into 2 years, but most programs are 3 years in length. Often, you’ll start off taking an engineering technician course, then with an extra year of work, you’ll upgrade your engineering technician program to an engineering technology program.

The engineering technologist then needs to go out and get an appropriate job, and learn for a couple years.

Eventually, they can apply to become a Certified Engineering Technologist. This will mean their education and work experience will be scrutinized, and they will either be approved or not. However; there is one final step to becoming an engineering technologist: The technologist must learn all the appropriate legislation, and memorize the code of ethics for their professional organization. They must pass the Professional Practice Exam, proving they know it.

Once the Professional Practice Exam is passed, and once they are accepted to become Certified Engineering Technologists, then they are given a certificate showing that they are certified engineering technologists.

To be a Certified Member means you’ve proven yourself in the field, as well as in the classroom. It also means that you’ve agreed to follow a strict professional code of ethics, and that you understand that violating that code of ethics could mean not just losing your certification, but facing fines from your professional association.

Unlike with Journeymen, there is no red seal program between provinces with your CET designation. If you will be working in a different province, you must transfer your membership to that province.

Different provinces also provide different opportunities to CETs. Alberta, for example, provides the opportunity for experienced Certified Engineering Technologists to gain the ability to practice professional engineering within a limited scope; something called a “P. Tech(Eng)”. Ontario is working towards a similar designation with the “LET” designation.

Certified Engineering Technologists who meet extremely stringent standards, and who are willing to undergo substantial upgrading and study can also be granted a P. Eng. in Ontario, a full license to perform professional engineering.

Thanks for reading!

Categories
Trades

The path of the Journeyman Instrument Technician

I’m Jason Firth.

A few days ago, I wrote about some examples of situations where using unqualified instrument techs caused catastrophic damage and the potential for a massive loss of life.

Today, I’m going to talk a bit about one of the paths a person can take to become a qualified instrument technician.

Centuries ago in Europe, there existed a system of guilds. These were organizations that controlled different skilled crafts and trades. They served two purposes: First, they served the public by ensuring that if a person was doing a trade, they had trained under different master craftsmen and had met a certain level of skill. Second, they served themselves by creating a monopoly in that craft or trade, which allowed them to charge more.

A person who wanted to enter a certain craft or trade would train as an apprentice under a master craftsman without charging a fee for their work, until that person had spent enough time working with their master, after which they would be released from their obligation, and they would become a Journeyman. In the German tradition, a Journeyman would then don the traditional garment, a black suit with a wide brimmed hat, and travel from master to master, learning the trade.

I want to spend a bit more time on this idea, because I find a lot of the aspects of it romantic. Some other journeyman traditions were that a Journeyman would wear a gold earring, to pay the gravedigger if they died. A journeyman would start their journey with 5 gold pieces, and after their journey was complete, they would have the same 5 gold pieces, to signify that the journey was not to become rich, but to learn their trade or craft. About half way through their journey (in terms of years), they would settle in with a master, and a few years after that, they’d begin work on the piece they would present to the guild; their masterpiece. This masterpiece would be the final test that convinces the guild that the journeyman is ready to become a master themselves. At that point, as a master tradesman or craftsman recognised by the guild, they could open their own shop in a town.

Today, some vestiges of that time still exist, but the guilds of antiquity are long disbanded.

Today, apprenticeship programs in Canada are managed by the provinces, each having its own legislation authorising the existence of such programs and describing how the program shall be administered. In addition, the 1994 Agreement on Internal Trade includes provisions for the “Red Seal” program, which allows a journeyman to travel anywhere in the country and maintain their certification freely and automatically.

Because each program is administered by a different body, there are differences in how each program is run, and the terminology used. The two most popular names are “Industrial Instrument Mechanic” and “Instrumentation and Control Technician”, although Alberta and the Northwest Territories call the certification “Instrument Technician”, and Saskatchewan and Nunavut call their certification “Industrial Instrument Technician”.

In most provinces, there are two paths to becoming a journeyman instrument tech.

The first path is called “apprenticeship”. In this path, a worker with no previous experience is hired as an apprentice. This person will learn from and assist the journeymen for 4-5 years. During the process, the journeymen will sign off that an apprentice has reached proficiency with a certain task, out of hundreds included in a book provided to the apprentice. Every year, there is a trade school the apprentice must attend for 10 weeks. The apprentice is often laid off for these 10 weeks, and won’t have their normal income during trade school. At the end of these 5 years, the apprentice must write a final trade exam. Depending on the jurisdiction, there can be multiple trade exams: One provincial, and one inter-provincial. (more on this later)

Although theoretically a person off the street may be hired for an apprenticeship, hiring a person for an apprenticeship is a gamble for the company. A company hiring an apprentice risks having spent time and money training an apprentice without getting a journeyman at the end, so they’ll hedge their bets. One way they’ll do that is hiring a person who already has shown a proficiency with instrumentation, by successfully completing an engineering technician or engineering technology program. Other times, a particularly bright operator at a union plant may get the first shot at an apprenticeship. It’s also common for journeyman electricians to apprentice as an instrument technician, since some of the skill sets align.

The second path is called “trade qualification”. In this path, a worker gets hired as an instrument tech through their education (an engineering technology diploma meets the academic requirements) and experience, and works alongside journeymen for 4-5 years. They must get their skill sets signed off by their supervisor. After working for 4-5 years and achieving the appropriate competencies, that person can write the trade exams the same way an apprentice can, and will be granted a full journeyman certification if they pass.

Occasionally, people use the term “interprovincial” and “provincial” with respect to their journeyman certification. Things vary between provinces, but generally there are different standards one must meet to achieve the provincial certification versus the interprovincial certification. In Alberta, for instance, an apprentice writes their provincial test first, then must write the interprovincial test later. In Ontario, for a long time, a person who got one score on their interprovincial exam (say, a 60%) was granted a provincial certification, but if they got another score (say, a 70%), they were granted an interprovincial certification.

There’s a variety of different categories of skills that an instrument tech must have to achieve journeyman certification.

Common Occupational Skills

Understanding of legislation and workplace standards in the workplace, including, but not limited to worker safety legislation, WHIMIS, PPE and lockout/tagout.

Process Measuring And Indicating Devices

There’s a huge number of instruments out there that a journeyman must understand and be capable of installing, maintaining, and troubleshooting. In addition, there are certain documentation and calibration standards which must be met to consider a calibration or validation valid, and certain tools a journeyman must have to assist in troubleshooting. When comparing similar trades, it’s this scope that differentiates the instrument tech while working on instrumentation.

Safety And Security Systems And Devices

There’s a huge number of safety related instruments out there that a journeyman must understand and be capable of installing, maintaining, and troubleshooting. In addition, there are certain documentation and calibration standards which must be met to consider a calibration or validation valid, and certain tools a journeyman must have to assist in troubleshooting.

Hydraulic, Pneumatic And Electrical Systems

One class that every instrument tech must take in trade school that no electrician will need to take is fluid mechanics. There are certain theories that must be understood for hydraulic and pneumatic systems to make sense. Pneumatics in particular may be a huge element of instrumentation — You might have an entire plant running on pneumatic controls, and an instrument tech must be capable of working on all of it. In addition, there’s the need to know electrical and electronic systems.

Final Control Devices

All the measurement in the world is meaningless if you can’t control something with that measurement. Instrument techs must intimately understand valves, actuators, positioners, variable speed drives, and all the various components that connecting them together. Fail safety is a critical part of this.

Communication Systems And Devices

Communication is a growing facet of instrumentation. Techs today need to understand control network systems and devices, signal converters, gateways, bridges, switches, and media converters. Recently the ISA asked: “Is instrumentation evolving into IT?”. I don’t think it is, because of all the other things that are involved, but the question is relevant considering the increased knowledge of IT required.

Control Systems And Process Control

This covers a wide scope, from discrete PID controllers, to PLCs, PACs and DCSes, to loop modelling and control fundamentals. There’s a number of different tools an instrument tech needs to understand, from feed forward and cascade loops, to ratio loops, to the different standards for tagging and description to ensure the next guy understands what you’ve set up.

There’s a lot to learn, and with all these areas of expertise comes a huge responsibility.

Thanks for reading!

Trade National Occupational Analysis – Instrumentation and Controls Technician

Categories
Process Control

Possibly Apocryphal

I’m Jason Firth.

A quote attributed to Mark Twain goes “History never repeats itself but it rhymes”.

I started my career going to college as an instrumentation engineering technologist. Over 20 years before that, my father was going to trade school as an instrument mechanic just a couple hundred kilometers away in Brandon.

Instrumentation is a fast moving trade: New computer control products are constantly coming out, there’s new technologies, new devices, new trains of thought, and tomes filled with the new ideas in control that come out every month. However, a lot of the fundamentals stay the same. A lot of the curriculum we both learned could be taught in either classroom. Pneumatics, electronics, fluid mechanics, op-amps, PID controllers, final control elements.

There’s one story in particular that both of us were taught, 20 years apart. It’s a story about why you should have qualified people working as instrument techs.

I’ll start with the process, and move on from there.

The Kraft paper making process starts with wood chips, which are then placed in a “digester”. A powerful caustic called “white liquor” is added to the digester, and the whole unit is heated and put under pressure. The white liquor dissolves the stuff keeping the wood fibres bound together, and once the digesting process is complete, you’ve got a combination of wood pulp, and spent white liquor, which is called “black liquor”, because it becomes filled with all the sugars and lignates and such from the wood. From the digesters, the result is placed on a giant drum called a washer, and the black liquor is washed out of the fiber, which then heads off to your paper machine or pulp machine or whatever you’re going to use the fiber for. The black liquor then pumped to the recovery process.

The recovery process takes that black liquor and “recovers” it into white liquor. The first step is that the black liquor is pumped into a giant boiler (we’re talking 8-10 stories tall, with a cross-section of a small house), called a “Recovery boiler”, where it burns. The sugars and lignates from the wood burn, producing heat. Once the liquor is burned, it drops into a chamber below the boiler, at which point it is now “green liquor”. From there, it gets sent to the recausticizing plant, where it is clarified and strengthened, and it becomes white liquor once again.

Recovery boilers are huge, as we’ve established, and they’re also quite high pressure. Plants I’ve worked at had steam of 800psi, but I’ve heard of plants as high as 1500psi. Besides producing enough steam to run the process, there is often enough steam left over to run a turbogenerator to offset the huge amount of electricity involved in the paper making process. I’ve seen turbogenerators of 20MW, but 100MW or more of electricity generating capacity is definitely out there. That’s enough energy to power a small city. When paper prices collapsed, some plants remained operational only from profits made by selling electricity back to the grid!

So you have these boilers that are dangerous by themselves simply by virtue of being massive pressure vessels containing enough energy to power a small city, but recovery boilers have an additional danger: The caustic which drops into the chamber below the boiler is called “slag”, and it reacts violently with water. Getting water into your recovery boiler is a great way to not have a recovery boiler any longer.

So finally, on with the story. Fort Frances is a town in northwestern Ontario, which for a long time had a pulp & paper mill. In recent years the mill has come on hard times, but before that it was in operation for decades.

The story goes, that on two separate occasions, they literally blew up their recovery boiler, because they weren’t using qualified people to handle their instrumentation and controls.

The first story goes like this: The union plant allowed someone from operations to work as an instrument technician without going through an apprenticeship first. One day, they installed a fail open control valve on the fuel line into the boiler. The first time the valve lost air pressure, the fuel valve opened 100%. The huge excess of fuel caused a boiler explosion. Fail safety is one of the fundamentals of instrumentation, so any qualified instrument tech should have caught the problem before it became a problem.

The second story goes like this: Apparently not learning from their first episode, the plant allowed someone who wasn’t qualified as an instrument technician to work as one. One day, the boiler was running dry to do some testing. This is an extremely unusual situation, and generally it isn’t something you’d ever do, because you can damage the boiler. No tag out was employed to explain that the boiler was being run dry intentionally, so when an operator noticed the low boiler water level, they panicked and started adding water. When you add cold water to a superheated empty boiler, the water immediately boils, turning to steam. The shock can cause an explosion. They ended up blowing up their boiler again. Lock out and Tag out are another one of the fundamentals of instrumentation, so any qualified instrument tech should have tagged the controller, preventing the problem.

In the age of the Internet, I haven’t been able to find any articles supporting the idea that either of these things happened. However, 20 years apart, at two different colleges, in two different programs, the same stories were told about the same mill in Northern Ontario, in both cases a cautionary tale of using unqualified instrument techs.

Thanks for reading!

Categories
Trades

The Power Destructitron X

I’m Jason Firth.

This post is meant to test embedding youtube videos into a post. This video is relevant to the blog because it is my term project from college: The Power Destructitron X.

The story of this surprisingly complicated project has two completely different morals. I think which one you decide on depends on your attitude.

As part of the instrumentation program, everyone has to take part in a term project. Each term project had a certain difficulty, tailored for a certain number of people working on the project. One guy was building a weather station. Another group was building a heat tester. Yet another was building an XY table.

The project I ended up being given was to create a controller to very precisely control a rotary table. I had a rotary encoder, and a stepper motor and stepper motor controller, and an inductive proximity switch. The basic theory was quite simple: Find a home position, determine the number of stepper motor counts that reached 100%, and control the number of counts, while paying attention to the binary encoder. I was going to do all the control in a modicon momentum PLC, and use Wonderware to display the data.

Well, I sort of jumped the gun. All the parts fit together beautifully, the programming was really easy, and I was basically done the assignment on the first day.

When I showed Mr. Shirtliffe, the teacher in charge of the instrumentation engineering technology program there for 30 years at that point that I had completed the project, He seemed to get pretty excited. He stared picking up random parts — a robotic hand here, a piece of an old laser printer there, a brutal looking 24vdc motor, and he gave me a new task: To build a “pick and place robot”.

Well, the project that was originally a difficulty level of 1, just became a difficulty level of 11. I went from having some nice low voltage, low current stuff, to having a huge variety of devices. There was suddenly now air, high current DC, AC, low current DC, I even ended up building some motor controllers from scratch…and I had to find some way to put them all together in a way that would somehow move a block around.

This isn’t hyperbole either; I was always in the classroom at 8am, but for weeks on end I’d stay in the lab until right before the last city bus that would take me home for the night. It was a huge amount of learning, research, work, and rework.

This video shows what I ended up with. Obviously this was before I had developed any real trade skills, but I’m still proud of the fact that I was able to somehow make a thing (no matter how contrived) out of these completely random parts.

(and no, this is probably not what I’d build today with an extra 8 years of design and field experience under my belt.)

As for the two different morals, it depends on your point of view: On one hand, there’s the saying that “the nail that sticks up gets hammered down”. That’s a perfectly legitimate way of looking at things: I would have gotten just as good of a mark if I’d kept my head down and pretended to be working on this thing that was already working, after all. However; I think of it this way: Instead of simply doing a fairly simple project that wouldn’t teach me that much, I got a chance to really stretch my legs, and learn first-hand the best way about a bunch of different controls.

Thanks for reading!