Nearly all virtually hosted ibiblio websites (anything sharing an IPv4 address with vhost.ibiblio.org) is currently unavailable while we investigate a security issue flagged by an outside entity. We do not yet have a time estimate for resolution.
Right now (about 1am Saturday July 25, EDT) access to “www.confluence.org” is hanging. (The server is not returning an error; instead, it’s just being non-responsive (like certain iBiblio staff 
Hi. For the second time today, I am allocating weekend hours away from home to the task of getting the above topic resolved. If and when I have time to resolution, I will supply it.
Thanks for looking into this. (I presume this is the cause of the “confluence.org” outage too.
1 Like
We have prepared a response internal to the University, and we have an articulated set of steps that must happen before the scope of the block on public access can be narrowed. Unfortunately, we still do not have a specific time to resolution, but we will continue to put in weekend hours toward that resolution and update again when we have more detail.
2 Likes
Thanks Cristobal, we’re deeply grateful for your work on this!
Sigi (OHP)
2 Likes
Additional staff are investigating system logs to help finish establishing the scope of impact. When that scope is settled, we can redeploy in a manner that allows unaffected vhosts to be publicly served. We unfortunately still do not have a specific time when we expect that to happen.
While we may have an additional information update later today (Monday the 27th), we will not be in a position to redeploy the vhosts today.
I suspect this is cold comfort to the many contributors whose collections have been unavailable since Friday afternoon, but we here have learned some valuable lessons and added additional tooling that should help us avoid a similar outage in the future. Please rest assured we have spent many hours between us on this issue, and that we will continue to work diligently until we are able to restore access to all impacted collections.
A range of collections will have their data rolled back to a known-good restore point, which unfortunately means some collections may have to re-upload or otherwise re-add content. If you are the point of contact for such a collection, expect for us to reach out to you by close of business Tuesday the 28th, Eastern time. Public access cannot be restored until that work is complete, and when we are more confident in a time estimate, we will give one.
1 Like
Any news for today? Hopefully you may have a better time estimate?
Could someone please fill us in on what is happening??
We had the unfortunate bad luck of having two different staff members, including myself, out sick. I am working again today, and will be as thorough and as prompt as I am able. I would love to give everyone more detail, but am unfortunately constrained in what I can share.
Good morning,
We believe that sites hosted under the infrastructure with the shared IP at vhost.ibiblio are ready for public access, and are now awaiting external review. We hope to have precise timing shortly.
1 Like
Good afternoon,
The campus-only restriction on vhost.ibiblio has been lifted. If your site is still unavailable, it may be because of either a change that we made to remove known-vulnerable software, or because we were unable to attest that we had done so. Please do reach out to us by opening a Help Request if your site does not load.
Good afternoon,
We are aware of a database issue on one of the database hosts that affected some collections shortly after our last post. We believe we have resolved that issue. As always, please let us know if something looks amiss.