Spork Boards

Boss Bitch

Simon's Avatar Picture Simon – December 18, 2007 07:38PM Reply Quote
For all your work woes.

El Jeffe – June 22, 2018 06:22AM Reply Quote
What a journey.
Quote
YDD
I have a problem.... my boss has told me to be nicer to a colleague on the other side of the country. Yesterday, I found out that said colleague either made a change or signed off on someone else's change which killed logging in our Test systems. If I'm right about this, then this was done several weeks ago, but they didn't notice. A roll out to production has been started, but fortunately a difference in configuration has meant that Production logging has survived. Actually, that difference in configuration is probably the reason for the failure of logs in Test - I discovered a nasty can of worms last year which must have been festering for at least two years before that. However, there were too many other fires at the time to fix it. I did warn the colleague about the issue I'd found - and suggested that they take advantage of the changes they were making (very closely related) to fix the issue simultaneously. Instead, it appears that another bale was cheerfully tossed onto the camel's back.

So.... what are the polite ways of suggesting that we perhaps have test systems for a reason?

WOW! So many of my days surround proper version AND change management, I can't even go into how ingrained this stuff is with my team and company. People get FIRED monthly for not following practices. As well, we highly segregate who/how/when folks can even TOUCH production and integrated (test) environments. Sadly, exceptions exist, and I even received an automated "ACCESS VIOLATION DETECTED BY YOU" email from our InfoSec security apps this week.

As for the personal side, the rules must apply to everyone or they work for no one. Everyone must understand and follow the reasons why the process exists. We even go so far as to have a software migration TEAM, that compares, migrates code from integration, fully tested (Aka QA) environment into production. We have two sets of weekly CHANGE ACCEPTANCE meetings to communicate and evaluate risk and opportunity - first at high level ( Executive team) Then on Thursdays at the technical teams... every mid to major impact change gets heard and everyone has a chance to ask/comment on every change in a conference call with dozens approaching 100 people at all levels of experience and expertise areas. We patch each month with windows, unix, database patches in non-prod. we have automated (sitting here right now looking at my daily check-outs) process (whole other TEAM I work with to build and operate these ) to test every non prod and production environments sometimes dozens of times a day. We have nearly FIVE production monitoring/alerting system; should those fail, we have three full-time humans that sit around the clock with EVERYONE's on-call and phone numbers to reach out to any technical team that needs to help resolve MAJOR outages/issues.

The process is NO LAUGHING MATTER and no one can afford to be negligent in their duties.

Again, simply put, if anyone is not doing the right things where I work, they are fired on the spot.

ARL (Moderator) – June 22, 2018 09:22PM Reply Quote
It's been a while since I've posted in here but hoo boy, someday soon do I have some stories to tell...

YDD – June 24, 2018 06:22PM Reply Quote
This is a product which is not officially dropped, yet everyone who can avoid doing work on it does so since management only every talks about the new-and-shiny product in meetings. Due to some inattentiveness on my part, I'm one of those stuck with maintaining the older product. The Powers That Be were not happy when someone (since departed) suggested that just keeping it running in-place would realistically take about eight people full time (in my more cynical moments, I feel that those who actively stopped touching it were well aware of the mess they had created).

In the past, the other group let some of their certificates get within 90 days of expiry, prompting all sorts of alarms to go off. And after a flurry of activity, they got commended for jumping right on the issue and fixing it. This prompted a colleague and I to wonder, not entirely inaudibly, if we should stop rotating our certificates before the alarms triggered.

johnny k – June 25, 2018 08:41AM Reply Quote
Quote

In the past, the other group let some of their certificates get within 90 days of expiry, prompting all sorts of alarms to go off. And after a flurry of activity, they got commended for jumping right on the issue and fixing it. This prompted a colleague and I to wonder, not entirely inaudibly, if we should stop rotating our certificates before the alarms triggered./quote]

Create a crisis, then fix it. A popular idea today. Certificate auth is one of my most-hated tasks, but since I've found that Let's Encrypt works for most of my needs, it's like magic.

Cloudscout – June 25, 2018 02:37PM Reply Quote
Det finnes ikke dårlig vær, bare dårlige klær!
Quote
El Jeffe
WOW! So many of my days surround proper version AND change management, I can't even go into how ingrained this stuff is with my team and company. People get FIRED monthly for not following practices. As well, we highly segregate who/how/when folks can even TOUCH production and integrated (test) environments. Sadly, exceptions exist, and I even received an automated "ACCESS VIOLATION DETECTED BY YOU" email from our InfoSec security apps this week.

As for the personal side, the rules must apply to everyone or they work for no one. Everyone must understand and follow the reasons why the process exists. We even go so far as to have a software migration TEAM, that compares, migrates code from integration, fully tested (Aka QA) environment into production. We have two sets of weekly CHANGE ACCEPTANCE meetings to communicate and evaluate risk and opportunity - first at high level ( Executive team) Then on Thursdays at the technical teams... every mid to major impact change gets heard and everyone has a chance to ask/comment on every change in a conference call with dozens approaching 100 people at all levels of experience and expertise areas. We patch each month with windows, unix, database patches in non-prod. we have automated (sitting here right now looking at my daily check-outs) process (whole other TEAM I work with to build and operate these ) to test every non prod and production environments sometimes dozens of times a day. We have nearly FIVE production monitoring/alerting system; should those fail, we have three full-time humans that sit around the clock with EVERYONE's on-call and phone numbers to reach out to any technical team that needs to help resolve MAJOR outages/issues.

The process is NO LAUGHING MATTER and no one can afford to be negligent in their duties.

Again, simply put, if anyone is not doing the right things where I work, they are fired on the spot.

Oh, I would love to sit down for a conversation with you about all of this. In a past life, I built a really good patch management practice. Right now, I've been handed responsibility for building an Evergreen Program to deal with the new release model that Microsoft is following for Windows 10 and Office 365. So many problems... primarily due to neglect and bad decisions by the leadership within our new Corporate Overlords. Mostly new leadership in place now but fixing everything that the Old Guard broke will take more time than we have.

Mokers (Moderator) – July 19, 2018 05:46PM Reply Quote
Formerly Remy Martin
As stated in another thread, I am starting a new job after 9 years. I really liked my old company, but had a chance to take on an IT director role somewhere. It's also a bit closer to the train, so I won't have to drive as muhc.

John Willoughby – July 19, 2018 05:57PM Reply Quote
Cyberdyne Systems Customer Support
Congratulations, Mokers! I hope that your new company will be just as great as your old one.

tomierna (Admin) – July 19, 2018 06:12PM Reply Quote
Hideously Unnatural
Congrats, Mokers!

ARL (Moderator) – July 19, 2018 07:16PM Reply Quote
Congrats - hope it's a good opportunity!

El Jeffe – July 20, 2018 06:56AM Reply Quote
What a journey.
Best of luck in your new role. Hiring? ;)

Ron Burns – July 20, 2018 03:11PM Reply Quote
"We look to Scotland for all our ideas of civilisation." Voltaire
Best of luck!

Ron Burns – July 20, 2018 03:12PM Reply Quote
"We look to Scotland for all our ideas of civilisation." Voltaire
Hope all goes well!!

Dave Loudin – July 21, 2018 07:33AM Reply Quote
Congrats!

johnny k – July 21, 2018 01:24PM Reply Quote
Easier commutes are worthwhile. Hope it's fun.

El Jeffe – September 12, 2018 05:56PM Reply Quote
What a journey.
25 year work anniversary.
I.am.old!

Sorry, only registered users may post in this forum.

Click here to login