Featured Product
This Week in Quality Digest Live
Innovation Features
Laurel Thomas
Trust your co-worker, but remember its video camera is running
Ben Brumfield
Taming the world’s loudest manufactured noise through engineering and perseverance
ISO
A third standard has just been published in this series
Nick Castellina
Technology maximizes human potential. It does not replace it.
Zara Brunner
Industry and government collaboration can foster advanced technologies and workforce development

More Features

Innovation News
Stereotactic robot helps identify target and deliver electrodes to target with submillimetric accuracy
Examine nine ways to help understand context, diffuse politics, heal relationships, and find the right work at the right time
Inkbit is overcoming traditional constraints to 3D printing by giving its machines ‘eyes and brains’
Deep Reality Viewer creates stereo, high-definition 3D images without using a monitor
Wing’s shape shifts automatically for aerodynamic loads (e.g., takeoff, landing, cruising)
Process concerns technology feasibility, commercial potential, and transition to marketplace
Identifying the 252 needs for workforce development to meet our future is a complex, wicked, and urgent problem
How established companies turn the tables on digital disruptors
Building organizational capability and capacity to create outcomes that matter most

More News

Shannon Brescher Shea

Innovation

How Project Management Made a Massive X-ray Light Source Possible

Ten years and $1 billion to build the NSLS successor

Published: Tuesday, July 23, 2019 - 12:03

Replacing a beloved tool is never easy. Erik Johnson had worked with the National Synchrotron Light Source (NSLS) for nearly 15 years when he and his colleagues began thinking about its replacement. But this switch wasn’t a matter of walking down to the hardware store.  

The NSLS, a Department of Energy (DOE) Office of Science user facility at Brookhaven National Laboratory, opened in 1982. Over 30 years, scientists—three of whom won Nobel prizes for their work—used its intense beams of light during the course of more than 55,000 visits to study atomic structures and chemical processes. Johnson came to the NSLS in 1985 as a post-doctoral student. By 2000, he and other leaders in the field realized the NSLS would soon be past its glory days.

They began dreaming up its successor: the NSLS-II. After five years of planning and research, the Office of Science approved the project to move forward.

“There was elation in the hallways,” says Johnson.

This was just the beginning. Ahead of Johnson and his co-workers stood a project that would take a decade and almost a billion dollars to build.

The Office of Science’s scientists and managers are familiar with this type of challenge. For decades, the DOE and its predecessors have been developing federal user facilities for scientists to probe the building blocks of the universe and of life.

Constructing these user facilities requires immense planning and coordination. Good project management keeps projects on time and on budget. The $912-million NSLS-II project put the Office of Science’s project management skills to the test.

Researchers required an X-ray beam 10,000 times brighter than the original NSLS.

Not a simple process

Institutions can’t take project management for granted.

DOE construction projects run through five stages. First, the Office of Science confirms the need for the project under the DOE’s mission. Then, the project team evaluates the alternatives and develops the basic design as well as an initial cost estimate for the approved alternative. Next on the docket is establishing the baseline—when managers set the project’s cost, schedule, and scope, instituting the standards against which they will measure the success of the project. Then, the DOE provides the go-ahead to start construction. The last step is certifying the project as complete.

A big machine comes with big challenges

With the NSLS reaching its technical limits, scientists needed the next big tool to study incredibly small materials under real-world conditions. To look closer than ever at subjects ranging from batteries’ chemical reactions to viruses’ structures, researchers required an X-ray beam 10,000 times brighter than the original NSLS.

To make the investment worthwhile, the team needed to design NSLS-II to be so advanced that it could stay at the forefront of science for more than three decades. It took more than 20 different scientific workshops to hammer out the requirements. Participants included both scientists who would use the facility and engineers who would design it.

“I had gone from 15 years where I knew everybody by first name and what they did to being surrounded by all new people,” says Johnson. “Which was pretty neat.”

To satisfy as many stakeholders as possible, the team designed the facility to immediately run at full capacity. They then scaled the plans down to what they could build initially. If they finished early or had funding left over, they could add pieces back. To make it possible to adapt the most advanced technologies, the team planned to add more research equipment and capabilities in phases after they finished construction of the main facility.

“[The process] was 90-percent preplanning and 10-percent execution,” says Robert Caradonna, the DOE Brookhaven Site Office deputy federal project director for the NSLS-II project.

All of that planning clarified the challenges that awaited them. 

Creating powerful X-rays requires massive machines that accelerate electrons to high energies. These machines use specialized magnets to control the electrons that produce the X-rays; equipment at experimental stations then harness these X-rays. Most facilities have rings that store the electrons; NSLS-II’s ring needed to be a half-mile around.

The building needed to meet several other specific requirements. Because temperature changes can influence the magnets’ size and position, the inside of the storage ring could never waver from a balmy 78°F. To ensure the beams would never tremble, the team needed to minimize the effect of vibrations from trucks on the nearby Long Island Expressway and waves from the Atlantic Ocean.

The project’s sheer complexity was perhaps the biggest challenge of all. The original schedule projected 5,000 separate activities. At the height of the project, the list expanded to 11,000. At some points, the team was managing a million dollars of work each day.

“We spent so much time with our heads down pushing on this thing that it didn’t really get to be overwhelming,” says Johnson. “We were too damn busy for it to get overwhelming.”

Prepared for disaster

In project management, being a pessimist can pay off. A huge part of ensuring the project ran smoothly was anticipating and managing risks.

The Office of Science purposely plans for the worst case scenarios. During the course of the project, the team ran more than 400 separate risk assessments. They also built numerous computer models that pinpointed exactly where every single piece of equipment needed to go.

One of the biggest areas of uncertainty was manufacturing the massive magnets. Buying 900 magnets anywhere is difficult. Buying 900 extremely high-powered, cutting-edge magnets from one supplier wasn’t going to happen.

“It would be nice if we could give it to one guy, and he could produce all the magnets,” says Caradonna. “We knew that was going to be impossible.”

There are only a handful of places in the world that could produce the magnets and other specialized pieces of equipment. The team developed seven contracts with five different suppliers, several of which were in other countries. The different languages, management cultures, geographic locations, and even measurement units caused conflicts. Some vendors were unresponsive.

To solve these problems, members of the NSLS-II team flew to the suppliers and consulted with them on site. The hands-on assistance improved communications and quality control. It even allowed NSLS-II staff members to get experience with the components before they arrived at the Brookhaven National Laboratory. Despite some early delays, the magnet manufacturing didn’t hold up the process as a whole.

The team was able to compensate for these delays because of savvy scheduling. In many projects like this, a delay in one step can cascade down to others, creating multiple delays and scheduling conflicts. In contrast, the NSLS-II team designed the project so they could change the order of the steps. For example, they built five identical pieces of the accelerator ring that fit together like LEGOs. If the magnets weren’t finished for one piece of the ring, they could still finish the others in time.

This approach also came in handy when the project received $150 million in funding earlier than planned through a special bill to help the country recover from the economic crash in 2008. With this funding and a favorable construction market, they negotiated a lower price with the construction company and finished the laboratory buildings nearly a year ahead of schedule.

“You never know when fortune is going to smile on you, and you never know when you’re going to do something sooner rather than later,” says Johnson. “It’s all about being ready.”

“It’s not the stuff that you build, it’s that what you build enables other people to do what’s important.”

Here a review, there a review

Preparation will get you only so far. The Office of Science’s regular and strict project reviews by independent experts got the team the rest of the way there. Every step of the process had a review, from the scope and scientific goals to the construction—54 total.

During the course of the project, the review teams gave more than 1,300 recommendations.

As the NSLS-II lessons learned document states: “Project reviews are the most important management tool to ensure the project is staying on track. If you are not required to have them, you should inflict them on yourself.”

NSLS-II is not the only project to benefit from rigorous reviews. The Government Accountability Office has cited the Office of Science’s reviews as a major reason why the majority of the Office of Science’s projects are completed on time and on budget.

The project teams aren’t the only ones who learn from the experience. The independent experts are often in charge of similar projects at their own agencies, national laboratories, or universities. 

10 years later

Ten years after the DOE approved the idea for NSLS-II, it was finished. The Office of Science declared it complete in March 2015, three months before its target date. It opened to users that July.

Owing to sound project management practices, there was enough funding available for the NSLS-II to include $68 million in optional features beyond the basic construction plan. It had an additional beamline to provide X-rays to another experimental station, a larger building that would make it easier to expand, and extra components to increase reliability. In 2016, the team won both the Project Management Institute’s Project of the Year award and the DOE Secretary’s Award of Excellence, the highest honor that the DOE awards to a project.

The Office of Science declared the NSLS-II complete in March 2015, three months before its target date.

“I was immensely proud, but fully cognizant of all of the work that needs to be done still to fully realize the potential of this instrument,” says Johnson.

Since it opened, the team has launched 28 experimental stations, or beamlines, out of a total of 60 stations it can support.

“Everything from the dreaming to the final delivery is going on at the same time,” says Johnson.

Despite all of the new machinery, one day the NSLS-II will become obsolete just like its predecessor. One Friday night, Johnson went home and noted that the old NSLS sign was still up. The next Monday, it was gone, replaced by a sign for the lab’s new Computational Science Initiative. To Johnson, that change reinforced the fact that his mission is about more than equipment. 

“Those people who used it, they still have those experiences,” he says. “It’s not the stuff that you build, it’s that what you build enables other people to do what’s important.”

First published May 28, 2019, on the Office of Science website.

The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information visit https://www.energy.gov/science.

Discuss

About The Author

Shannon Brescher Shea’s picture

Shannon Brescher Shea

Shannon Brescher Shea is a senior writer/editor in the Office of Science’s Office of Communication and Public Affairs. As the content team lead, she writes and edits feature stories covering the Office of Science’s discovery research. She also manages the Science Public Outreach Community for communicators in the fundamental research area. Previously, she was a communications specialist in the Vehicle Technologies Office in the Office of Energy Efficiency and Renewable Energy. She managed publications, websites, and social media for both Clean Cities (the alternative fuels deployment program) and the Office’s research and development portfolio.