Redo
Ever heard the term “GIGO”? If not, it means “Garbage In, Garbage Out”. That’s exactly what has happened to the SQL data warehouse I worked over a year on. The main data source, a student information system (SIS), is so fubar that I can’t guarantee the data.
The problem stems from lazy end users (about 80) of the SIS. Instead of checking if a student already existed, or heeding the error that a student already existed, they would just change some small detail about the student. This little “work around” causes duplicate entries in the SIS. Trying to code around the problem is out of the question simply because there are too many variables. Plus, when the duplicates are found in the SIS, I wouldn’t know which entry is correct so I may import the invalid entry while ignoring the valid one.
This sucks on so many levels. The highest level is the fact that the best option is to blow away what I have and start over. That will involve not only importing information from the SIS but also importing every standardized test (ACT, SAT, etc.) again in order to match it to a student.
I have brought this issue to the attention of those in charge but it was pointless. The answer is always “Maybe they just need more training”. Yeah, that’s it. The other four times they were trained to not do it was wrong. Fifth times a charm.
Idiocy - Never underestimate the power of stupid people in large groups. - Despair, Inc. Idiocy poster
25.Jul.08 
Microsoft SQL
You can leave a response, or trackback from your own site.
2 Responses to “Redo”
Leave a Reply
- wyckedone.net
F/OSS
Free Antivirus
Free Firewall
Free Utilities
Links
Tech Articles
Tech Boards
This site runs on
I think some social engineering is needed. How about putting an error message like “Do NOT create duplicate entries for students”?
I am currently in the process of inventing an automatic Crotch-Puncher. For the small price of ONE MILLION DOLLARS, I will let you lease it for a year.
It comes with a customizable open source (of course) setup configurer and USB thingamabob.
If this is successful, I plan on expanding my range to include the patent-pending Crotch-Kicker (complete with bluetooth and wifi capability).
Basically you set it up so that if they do something you specifically program/tell/error-message them NOT TO DO, they get punched in the crotch.
The best part about using this product, is that the errors are self correcting. Even if they become numb to the agonizing pain, you can rest assured that they won’t be breeding.
I await a response with infinite patience.
Here is another thing to worry about. If those 80 nutjobs messed up big, how can you tell which record is correct? I am thinking of fields like address and telephone.
The reason I asked is because I changed my address and cell number since I started uni, and if they have to start over from the (I am guessing) last year, then they pretty much won’t have any contact details. (At least non which are current).
If I understood the post that means that someone is going to have to do a lot of grunt work and paperwork and make sure that the records are correct.
Was this what happened here, or am I misunderstanding the whole situation?
If I understood the post that means that someone is going to have to do a lot of grunt work and paperwork and make sure that the records are correct.
In a nutshell, you nailed it. The biggest issue concerns all of the test scores I’ve already stored using a student number. If the student number assigned to the test (ACT, for example) is the wrong one, I will have to update the table to show the correct number. That’s not a big deal when it’s one or two numbers. It’s a problem when you talk about a few hundred.
That’s why rebuilding may be my best option. I can import a thousand test scores in less than 30 seconds. That would assign the corrected number.
A newer version of the SIS stops these duplicates from occurring. Unfortunately, the district can’t afford the upgrade at this time because it involves buying new web and SQL servers (5,000+ users, all web based).