Author name: jrc

Jul 10, 2015 @ 3:40PM We are currently experiencing some service outages and are working to resolve the problems. Email is currently one of those services not working. More updates to come soon…

Jul 10, 2015 @ 4:05PM We have located the cause of the problem and services are starting to get restored now.

As part of the network relocation work, we will need to bring down the
wireless network in the CS Building. This work will take place
6:00am-9:00am tomorrow morning (1/31/2014).

Users may have received a phishing attempt to their CS e-mail account. The subject may be \”Account Login Alert!\” or \”Princeton Security\”.

Please be aware that there seems to be a coordinated spear-phishing campaign targeting the CS Department. Various e-mails are being delivered to accounts making assorted claims from improper password use to free money or local news and events.

Remember to always exercise caution with e-mail and web sites. These messages prey on your inattention in order to steal your credentials and use them for various nefarious purposes. DO NOT CLICK SUSPICIOUS LINKS, and DO NOT TYPE YOUR USERNAME OR PASSWORD on any site that you are not 100% sure belongs to the CS Department.

If you are not sure, please feel free to forward suspicious messages to CS Staff (including full headers if you can) and we can verify whether or not the message is valid.

If you may have compromised your account, contact CS Staff

If you need any additional assistance, please contact the CS Staff at any time.

Both of the CS LDAP authentication servers went down this morning due to hardware failure. The LDAP servers provide authentication for e-mail, SSH access to public CS servers, and the CS file system. We are working to restore the servers. We currently don\’t have a time when the service will be restored. Updates will be posted to this site as we make progress repairing the servers.

8:30 AM Update – The servers are now booting again and we are working to restore the LDAP software and the LDAP database.

9:20 AM Update – The LDAP software is now working and we are working on restoring the LDAP database.

10:20 AM Update – The LDAP database has been restored from an older version dating back to September. We are trying to update to the latest backup from last night.

11:11 AM Update – We were unsuccessful in restoring to a later version of the LDAP database. If you created a CS account or changed your password after September 24, 2013, you will need to contact CS Staff to update your password.

Who is affected:
All users of the CS Department Network

What is happening:
The Department\’s primary border firewall will be replaced with a new firewall device. During the replacement, connections from the CS Department to the main campus, the internet, or to portions of the CS Department network which are outside our firewall, will be interrupted. However, as OIT will also be performing router maintenance between 05:00 and 07:00 (see earlier announcement), we don\’t anticipate this work causing much additional impact.

Why is it happening:
This is the second step of our three-step process to provide 10Gbps networking from our network core out to the main campus, and, by extension, the internet. This upgrade will give our firewall the internal capacity to support 10Gbps connections.

Due to technical complications, the decision was made to postpone the work in order to minimize the risk of problems. Those issues having been worked out, we have rescheduled this maintenance for Monday, August 12, 2013.

Who is affected:
Potentially, all users of CS Department computing and storage infrastructure.

What is happening:
On Thursday, the storage cluster that provides file service for all CS Department public services will undergo upgrades to its network interfaces.

During this maintenance, connections to Samba/CIFS file shares will experience occasional, temporary interruptions. Users may need to re-establish connections to the file share in some cases.

We DO NOT anticipate actual downtime for other services, but since all maintenance involves the risk of broader problems, this announcement serves as a heads-up. In the event of problems with this maintenance, impacts could range up to outages of ALL CS Department public services that rely on the storage cluster (file service, public login hosts, e-mail, web service, etc). Note that wired and wireless networking connectivity will not be affected.

Why is it happening:
This upgrade will provide 10Gbit network connectivity to our storage cluster, getting us one step closer to end-to-end 10Gbit capability for our services.

We are working to resolve an issue with DropBox. The site is currently down. We will post status updates here.

Update 11:39 AM – DropBox is now working again. Please let us know if you have any issues.

The main CS website, virtual websites, wiki and CS Guide are currently down. We are working to correct the problem.

UPDATE 10:06am: we believe we have isolated the issue and continue to correct the problem.

UPDATE 11:03am: all systems are back up and working as expected.

From http://helpdesk.princeton.edu/outages/view.plx?ID=3906

\”OIT network administrators have informed the Help Desk that the OIT provided network is experiencing problems acccessing resources off of the Princeton campus. This includes all internet websites as well as mail resources. Connections to intranet resources (file shares, OIT mail servers, Princeton websites) will all still work from on campus connections. OIT administrators are working to resolve this issue and more information will be provided here as it becomes avaiable.\”

As of 4:39pm today both the CS and HPCRC external fiber connections are currently down. We are working with OIT to restore the connections and will post updates as we know more. All external traffic outside of the CS network will not work because the fiber connections are down.

UPDATE 8:02pm: we now have the external connection restored to the main CS router and have connectivity to the outside from the CS department. We still have no connections to HPCRC.

UPDATE 8:36pm: we now have the HPCRC fiber connection restored.

UPDATE 8:40pm: All network connections have been restored and connectivity to the outside should be working as expected.

EPILOG 9/26/2012 11:21am: Root cause may have been related to renovation at another site where our fiber is routed.

– CS Staff