DoIT Alerts

Update @ 6:37 PM: After a rolling restart was performed, course menus appear to be consistently stable and content is available. Users also report the error messages are no longer appearing. 

At approximately 10:00am on October 29th, UMBC users began reporting error messages upon attempting to access Blackboard courses. Instructional Technology staff opened a ticket with Blackboard Support and are continuing to monitor the service disruption.

As of 11:15am, this issue remains under investigation with Blackboard support.

Users may observe missing items from course menus including attendance, announcements, groups, and messages. Users who are able to log in courses may experience degraded performance and difficulty accessing course content and assessments. Most users are able to log into Blackboard itself and the issue is not system wide at this time.

Clearing the browser cache may offer temporary relief for the issue. 

If you encounter an issue when accessing a Blackboard course, please open an RT ticket and provide the session debug code, if available. The unique error id can be shared with Blackboard support to help clarify the errors observed in your course(s). Screenshots are also helpful.

This post will be updated with additional information as we receive it.

• Check our extensive FAQ collection 
• Open a ticket via RT
• Follow the Instructional Technology & DoIT myUMBC groups
• Request a consult with instructional technology staff
• PIVOT | Academic Continuity | Keep On Teaching | Student Technology Resources

• More Information about RESOLVED - Blackboard Service Disruption - Course Content Access

At 6:40 PM on September 22nd, Blackboard reported a system performance issue affecting clients in the US East region. This disruption may prevent users from accessing the Blackboard platform. Users who are able to log in may experience degraded performance and difficulty accessing course content and assessments. 

As of 11:00 PM, system performance issues remain unresolved.

Blackboard engineers are working to address the issue as quickly as possible. Instructional Technology staff opened a ticket with Blackboard Support and are continuing to monitor the service disruption.

Update: 

As of 3:15 am on September 23, Blackboard reports that this incident has been resolved. Please open an RT request for any continuing concerns related to system performance.

This post will be updated with additional information as we receive it.

• Check our extensive FAQ collection 
• Open a ticket via RT
• Follow the Instructional Technology & DoIT myUMBC groups
• Request a consult with instructional technology staff
• PIVOT | Academic Continuity | Keep On Teaching | Student Technology Resources

• More Information about Resolved - Blackboard Service Disruption - System Performance

At approximately 3:42pm, DoIT staff became aware of a potential outage with third-party tools on our instance Blackboard Learn. System admins confirmed reports that an AWS outage in US-East is impacting Blackboard and many third-party tools in Blackboard including, but not limited to:

• Class Collaborate
• VoiceThread
• Cengage
• Pearson
• CourseArc
• WileyPLUS

The service disruption prevents users from authenticating into an application, whether through Blackboard or directly via the vendor's website. Active sessions may also be disrupted. Inline grading and viewing student submissions is also affected by this outage.

AWS is responding to the incident and will provide further updates to vendors about the service disruption. Instructional Technology staff opened a ticket with Blackboard support and continue to monitoring the outage.

This post will be updated with additional information as we receive it.

Update 5:15 PM - Many AWS cloud services have been restored. Amazon continues to work on the issue. Blackboard appears to be functioning normally, but Instructional Technology staff will continue to monitor server performance for the next 24 hours.

• More Information about RESOLVED: AWS Outage Impacting 3rd-Party Tools in Blackboard

I wanted to thank everyone for the quick feedback as we worked through WiFi Issues on Monday that affected several lecture halls and large classrooms. 

We made changes to the wifi devices in these spaces throughout Monday to resolve the connectivity issues we saw in the morning and afternoon. The problems were largely due to a problem with the network not automatically scaling in response to the demand. Our staff was able to troubleshoot the problem and make a number of adjustments to the system late in the day. These adjustments allowed us to force the devices to increase their power and capacity, ensuring everyone could access the wireless during their classes. 

Throughout Tuesday and Wednesday we met with faculty just before class to check in, and had staff sitting within the classes to monitor and check on the adjustments we had made. We observed much better performance, and confirmed with faculty and students on both days that they were able to access the network during class without issue. 

While I regret that the issues on Monday occurred, I really appreciate the cooperation and willingness to  help us get this resolved quickly. We are going to continue monitoring classes in these spaces to ensure everything is working smoothly over the next week. I would encourage anyone who experiences issues to submit a ticket via myUMBC Help so we can address your issue as quickly as possible. I'm also happy to provide more details on the changes and adjustments we made to address this issue if folks are interested, my email is listed below. 

To the entire UMBC community, thank you for all of the communication, feedback and partnership getting this resolved. I'm sorry for the inconvenience it caused. 

Damian Doyle

damian@umbc.edu

Senior Associate VP and Deputy CIO

DoIT - UMBC

As of around 4pm all campus technology applications and services were fully restored. We verified our power backup systems were fully operational again as well.