Nothing to be done
Oct. 21st, 2003 12:35 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
j4I only came into the office yesterday because they wanted me to do some testing on the alpha release of Early English Bo(ll)o(c)ks Online, and when I got there, it turned out that the development server had crashed so nothing was available for testing. However, I was in then, and when I'm in the office people have a tendency to come sidling up to me with muttered requests for perl scripts, or "perlscript routines" as one girl calls them. (Perlscript routines are like routines except that they don't run on DOS, so you have to run them on telnet.)
Now the problem is not that these people ask for difficult things. Mostly what they want turns out to be fairly trivial, if you can ever work out what it is that they want. The problem is not even that they tend to be asking for scripts to do things that you just shouldn't want to do. That's manageable: ours not to reason why, ours just to look faintly puzzled and say "But that's what you asked for" when it does the wrong thing. No, the real problem is that these people don't just move the goalposts, they run around like headless chickens with the bloody goalposts strapped to their backs.
"Can you write a script to match these SGML records against these other records, by ID number?"
"Sure. Details?"
"Well, the first lot of records are ... er ... here. There's one record per file, so that's about 200,000 files."
"Gosh."
"The second lot are here. They're all in one file, one citation per line, so that's about 25,000 lines. They're in a peculiar pseudo-MARC format."
"Okay. And there's an ID in the first lot that matches an ID in the second lot?"
"We-ell... sort of. There's this field in the first lot, which matches the stuff that follows ËPMID. in the other lot."
"Ah-huh..."
"The field ends when you get to the next Ë."
"And this field is in every record."
"Yeah. Er, except when it isn't."
"Right."
"Sometimes you have to just match the title, which should be nearly the same in both."
"Mm."
"And if the title doesn't quite match, you'll need the routine to check the author as well, you know, to make sure. Though the name forms might be different."
"Ngghh."
"And if they do match, or if they sort of match, you probably want to check the date as well."
"Nn."
"The dates in this lot are in a standard 8-digit thing."
"And the dates in the other lot...?"
"... aren't."
"..."
"And the output, I mean the records which might match, can they be sorted in order of, like, how likely it is they match?"
"Well, you'll have to define what that means."
"Yeah. Well, it doesn't really matter, it's just, like, if we could add up how many fields match, and how much they match, and then sort them in order of that."
"I'll ... see what I can do."
"Oh, and, I meant to ask... it would be really helpful if before doing this, you could write a script to sort the original records into separate files by journal title."
"Er..."
"There's an ID number which corresponds to the journal."
"Right."
"Or, actually, if you could sort them into files by journal title and issue number, that would be good."
"Right. Will they all have an issue number?"
"Er ... no."
"Will they all have a journal ID number?"
"Er ... probably."
"Right. So, 200,000 files sorted --"
"-- oh, sorted by publication date within the files, yeah, please."
"... sorted into separate files by journal title and issue number, within which they're sorted by date."
"Yeah, because that way it'll make it much easier for the freelancers to start matching records by hand while you're writing this script."
"Gggggghhhhhhhkkkkk."
Back in again today, so that I can get that testing done; the dev server is now fixed, but Magic Geoff has to fix all the other stuff that needs fixing before anybody can test the alpha, which he couldn't do yesterday because the server was down... so I still have nothing officially to do, which means people still keep asking me for crap perl scripts.
This job is even more dead-end than it was before I "quit". It's not even adding anything to my CV any more, except time. ("Except my life. Except my life. Except my life.")
"What is there in the bag?"
"Sand."
Now the problem is not that these people ask for difficult things. Mostly what they want turns out to be fairly trivial, if you can ever work out what it is that they want. The problem is not even that they tend to be asking for scripts to do things that you just shouldn't want to do. That's manageable: ours not to reason why, ours just to look faintly puzzled and say "But that's what you asked for" when it does the wrong thing. No, the real problem is that these people don't just move the goalposts, they run around like headless chickens with the bloody goalposts strapped to their backs.
"Can you write a script to match these SGML records against these other records, by ID number?"
"Sure. Details?"
"Well, the first lot of records are ... er ... here. There's one record per file, so that's about 200,000 files."
"Gosh."
"The second lot are here. They're all in one file, one citation per line, so that's about 25,000 lines. They're in a peculiar pseudo-MARC format."
"Okay. And there's an ID in the first lot that matches an ID in the second lot?"
"We-ell... sort of. There's this field
"Ah-huh..."
"The field ends when you get to the next Ë."
"And this field is in every record."
"Yeah. Er, except when it isn't."
"Right."
"Sometimes you have to just match the title, which should be nearly the same in both."
"Mm."
"And if the title doesn't quite match, you'll need the routine to check the author as well, you know, to make sure. Though the name forms might be different."
"Ngghh."
"And if they do match, or if they sort of match, you probably want to check the date as well."
"Nn."
"The dates in this lot are in a standard 8-digit thing."
"And the dates in the other lot...?"
"... aren't."
"..."
"And the output, I mean the records which might match, can they be sorted in order of, like, how likely it is they match?"
"Well, you'll have to define what that means."
"Yeah. Well, it doesn't really matter, it's just, like, if we could add up how many fields match, and how much they match, and then sort them in order of that."
"I'll ... see what I can do."
"Oh, and, I meant to ask... it would be really helpful if before doing this, you could write a script to sort the original records into separate files by journal title."
"Er..."
"There's an ID number which corresponds to the journal."
"Right."
"Or, actually, if you could sort them into files by journal title and issue number, that would be good."
"Right. Will they all have an issue number?"
"Er ... no."
"Will they all have a journal ID number?"
"Er ... probably."
"Right. So, 200,000 files sorted --"
"-- oh, sorted by publication date within the files, yeah, please."
"... sorted into separate files by journal title and issue number, within which they're sorted by date."
"Yeah, because that way it'll make it much easier for the freelancers to start matching records by hand while you're writing this script."
"Gggggghhhhhhhkkkkk."
Back in again today, so that I can get that testing done; the dev server is now fixed, but Magic Geoff has to fix all the other stuff that needs fixing before anybody can test the alpha, which he couldn't do yesterday because the server was down... so I still have nothing officially to do, which means people still keep asking me for crap perl scripts.
This job is even more dead-end than it was before I "quit". It's not even adding anything to my CV any more, except time. ("Except my life. Except my life. Except my life.")
"What is there in the bag?"
"Sand."
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2003-10-21 05:06 am (UTC)no subject
Date: 2003-10-21 05:31 am (UTC)$arg= shift || die "No search string provided.\n";
print "No match found for $arg.\n";
no subject
Date: 2003-10-21 06:12 am (UTC)If you can actually get something to do even parts of that written, I'd've thought that you could call yourself a professional-quality Just Another Perl Hacker and go in for more Perl-ish jobs if you wanted them (which, from what you've been saying, I presume you don't).
no subject
Date: 2003-10-21 06:22 am (UTC)Besides, very few people employ people just to write Perl, and Perl's all I can write. And only bad Perl at that, and only very specific types of bits and pieces of bad Perl.
I am not a programmer. I never want to be a programmer. And not only because I've seen what it turns people into.
no subject
Date: 2003-10-21 06:39 am (UTC)no subject
Date: 2003-10-21 07:13 am (UTC)"I want the database to know what I want it to do, and do that."
Not saying anything about the lack of telepathy modules in PHP was among the hardest things I've done this year.
*hug*
no subject
Date: 2003-10-21 04:17 pm (UTC)Ex-actly. Computer is magic!
The problem is, like most of the things they want done, it's something that a human being finds relatively easy. "Does this thing look like this other thing?" isn't a very difficult problem for an ugly great lump of biocomputer to solve, so I guess it's understandable that it should be hard for said ugly great lump to see why it may be more difficult for a computer. (And software engineers aren't always guiltless of perpetuating this problem by propagating the idea that what they do is Magic of the Highest Order -- victims of their own illusion, or something.)
Of course, it's still REALLY FUCKING FRUSTRATING, though. If our desks were more chewable mine would have lost its legs a long time ago. <sigh>
no subject
Date: 2003-10-21 03:22 pm (UTC)"Sand."
[He jerks the rope.] "On!"
no subject
Date: 2003-10-21 04:18 pm (UTC)no subject
Date: 2003-10-22 09:44 am (UTC)