User:Nejd
The following code is a Perl program I wrote, which I call "CnvBRtbl2ntrmd.pl". If you copy this code to some folder of your harddrive, store it as a file called "CnvBRtbl2ntrmd.pl", then create the other Perl program file called readtemplate.pl, and if you have Perl on your machine, you can very easily create the wikiscript needed for the latest version of the 2010_NL_Record_vs._opponents. The 'latest' version, that is, based on the Head to Head Records as stored at http://www.baseball-reference.com/leagues/NL/2010-standings.shtml. The web site BASEBALL REFERENCE maintains a bunch of baseball related statistics and tables. The input for this Perl program is a table called "Head-to-Head Records" and is located on this page: http://www.baseball-reference.com/leagues/NL/2010-standings.shtml. You just want to copy the text part of the table (the first and last lines of this table starts with a list of the National League team abbreviations). This text file is then used as the input file for this Perl program. If you have Perl on your computer, you can create an intermediate file (used as input for readtemplate.pl) in the following manner. Let's assume you have copied the current Head-to-Head Record table into a text file called BR_HtoH_July26.txt on your harddrive in a folder called "MyWikiWork" on your C: drive. At a command prompt, you would type something like the following:
C:\MyWikiWork> perl CnvBRtbl2ntrmd.pl BR_HtoH_July26.txt > July26_intermed.txt
This assumes that you have Perl available on your computer, and it has been set up to be accessible from this folder. After executing the preceding command line, you would then have your intermediate file, called July26_intermed.txt, which will then be the first of two input files for the second Perl program called readtemplate.pl, which is what ultimately creates the new wikiscript that you can use at http://en.wikipedia.org/wiki/Template:2010_NL_Record_vs._opponents.
Nejd (talk) 15:40, 27 July 2010 (UTC)
CnvBRtbl2ntrmd.pl
# Here is a PERL program I have written to convert the table (titled "Head-to-Head Records") found here:
# http://www.baseball-reference.com/leagues/NL/2010-standings.shtml
# into an intermediary file. This intermediary file is then to be used as the first input file of my
# other PERL program named "readtemplate.pl", which, using the 2nd input file which is a copy of the
# wikiscript used to describe the table at http://en.wikipedia.org/wiki/Template:2010_NL_Record_vs._opponents,
# will output a modified version of the wikiscript description of template 2010_NL_Record_vs._opponents
# which now incorporates the records found in the original "Head-to-Head Records" table.
#
# I first started working on this PERL program on September 23, 2008 and spent several days working on it.
# On April 26 and 27, 2009, I did further work to handle situations I had not forseen when I worked on
# it last September. These being, unentered values within the table, and the possibility of the first
# column labeled as "AvgOppRec" not being included in the table. So I guess I will call this version 2.
$numberofteams = 0; # this will be set by the number of team abbreviations listed in the table header
$headerRead = (0==1); # flag to indicate if table header has been read. Initialize to FALSE
$hasAvgOppRec = (0==1); # Does this table start with the AvgOppRec column? Initialize to FALSE
while (<>) { # see page 80 of "Learning Perl on Win32 Systems" book
if (/Interleague Records/) { # use existance of this string to indicate table header line
if ($headerRead) { # if header line has been read and we are now reading the same line on
last; # the bottom of the table (which I am assuming exists), exit while loop
}
# 1st line of table looks like this:
# |AvgOppRec|ATL FLA NYM PHI WSN|CHC CIN HOU MIL PIT STL|ARI COL LAD SDP SFG Interleague Records
# or this:
# |ATL FLA NYM PHI WSN|CHC CIN HOU MIL PIT STL|ARI COL LAD SDP SFG Interleague Records
if (/\|AvgOppRec/) { # if this is a table whose first column is labeled "AvgOppRec" then
s/\| *AvgOppRec *//; # delete that label (and preceding '|') so the following '|' is
# now in the same position as was the original first '|'
$hasAvgOppRec = (0==0); # set $hasAvgOppRec to TRUE
}
s/Interleague Records//; # we assume that the Interleague Records column is at the end
#### beginning of code to deal with case where some table values not entered (will assume 0)
### this creates an array @teamcolpos which is to hold the last char position of every team name
## relative to the left-most '|' after the "AvgOppRec" column (if it exists) has been eliminated
#
$header = $_; # save this header line before we start doing further modifications
$frstclmndx = index($_, "|"); # get position of 1st '|' indicating left-delimiter of 1st column
s/[A-Z][A-Z][A-Z]/XX@/g; # change all team names in team header line into "XX@"
$i = 0;
while (/XX@/) { # while there are still some team names remaining
$ndx = index($_, "@"); # position of first last-character-of-team-name
$teamcolpos[$i] = $ndx - $frstclmndx; # store in array
s/@/X/; # rename team whose character position we just found from "XX@" to "XXX"
$i++;
}
$_ = $header; # restore value of $_ to what it was at start of this little diversion
# printf($_);
#
##
###
#### end of code to deal with case where some table values not entered (will assume 0)
s/\|/ /g; # replace '|'s with SPACEs
s/^ *//; # remove any SPACEs found at beginning of this line
s/ *$//; # replace zero of more SPACEs at end of line with nothing
chomp($_); # get rid of newline at end
# print "\$\_ = '$_'\n";
# $_ now looks like this: "ATL FLA NYM PHI WSN CHC CIN HOU MIL PIT STL ARI COL LAD SDP SFG"
@teamsarray = split; # convert list of SPACE-separated team abbreviations stored in $_ into array
$numberofteams = @teamsarray;
# print "Number of teams = $numberofteams\n";
for ($i=0; $i<$numberofteams; $i++) {
$teamsarray[$i] = sprintf("$teamsarray[$i],%02d", $i);
}
# @teamsarray now looks like this (I am using semicolons to separate elements of array here):
# (ATL,00;FLA,01;NYM,02;PHI,03;WSN,04;CHC,05;CIN,06;HOU,07;MIL,08;PIT,09;STL,10;ARI,11;COL,12;LAD,13;SDP,14;SFG,15)
@sortedteamsarray = sort(@teamsarray);
# for ($n = 0; $n < $numberofteams; $n++) {
# print "Team $n = '$sortedteamsarray[$n]'\n";
# }
# print "(";
# for ($n = 0; $n < $numberofteams; $n++) {
# print "$sortedteamsarray[$n]";
# if ($n<($numberofteams-1)) {
# print ";";
# }
# }
# print ")\n\n";
# @sortedteamsarray now looks like this:
# (ARI,11;ATL,00;CHC,05;CIN,06;COL,12;FLA,01;HOU,07;LAD,13;MIL,08;NYM,02;PHI,03;PIT,09;SDP,14;SFG,15;STL,10;WSN,04)
$headerRead = !$headerRead; # header line is repeated on bottom of table
}
elsif (/^ *[A-Z][A-Z][A-Z]/) { # only match lines that begin with 3 capital letters, such as "ATL" or "ARI" or "NYM"
# first line of table proper looks something like this:
# ATL| 74-76 | -- 10 9 2 6| 0 3 2 3 2 2| 5 4 4 5 2|LAA:2-1,OAK:2-1,SEA:2-1,TEX:1-2,TOR:1-2
# or (if there is no AvgOppRec column) like this:
# ATL| -- 10 9 2 6| 0 3 2 3 2 2| 5 4 4 5 2|LAA:2-1,OAK:2-1,SEA:2-1,TEX:1-2,TOR:1-2
# or at beginning of season, like this:
# ATL| -- 0 2 4| 1 | |
if ($hasAvgOppRec) { # if there is an AvgOppRec column
s/\|[^|]+\|/\|/; # replace everything within and including the first 2 '|'s with one '|'
}
#### beginning of code to replace non-entered values with zeros, using array @teamcolpos
###
##
#
$frstclmndx = index($_, "|"); # get position of 1st '|' indicating left-delimiter of 1st column
$thisline = $_;
for ($i=0; $i<$numberofteams; $i++) {
if (substr($thisline, $frstclmndx+$teamcolpos[$i], 1) eq " ") {
substr($thisline, $frstclmndx+$teamcolpos[$i], 1) = "0";
}
}
# print($thisline);
$_ = $thisline;
#
##
###
#### ending of code to replace non-entered values with zeros, using array @teamcolpos
# s/ ./,/g; # remove all spaces
#
# if ($hasAvgOppRec) {
# ($team,$AvgOppRec,$wins[0],$wins[1],$wins[2],$wlALstr) = split(/\|/, $_);
# $_ = $AvgOppRec; s/ //g; $AvgOppRec = $_;
# } else {
# ($team,$wins[0],$wins[1],$wins[2],$wlALstr) = split(/\|/, $_);
# }
($team,$wins[0],$wins[1],$wins[2],$wlALstr) = split(/\|/, $_);
$_ = $team; s/ //g; $team = $_;
for ($i = 0; $i <= 2; $i++) {
$_ = $wins[$i];
# s/^ *//; # get rid of leading SPACEs
s/--/%/; # replace '--'s with '%'s
# s/(\d) +(\d)/$1,$2/g;
$wins[$i] = $_;
}
$brwins = $wins[0].$wins[1].$wins[2]; # just concatenate the wins against all 3 divisions
# print("team = '$team'; AvgOppRec = '$AvgOppRec'\n");
# print(" wins-east = '$wins[0]'\n");
# print(" wins-central = '$wins[1]'\n");
# print(" wins-west = '$wins[2]'\n");
$_ = $brwins; s/^ *//; $brwins = $_; # get rid of leading SPACEs
# print("\$brwins = '$brwins'\n");
@brwins = split(/ +/, $brwins); # @brwins is now array of wins for each team, based on baseball-reference
# the following for loop changes the sequence of teams against which the current $team's wins are listed
# from the sequence as it is stored in baseball-reference (division east teams followed by central followed by west)
# to alphabetical order (the way wikipedia's table does)
# The originally sequenced array is @brwins, the newly sequenced array is @wins
for ($i=0; $i<$numberofteams; $i++) {
($TEAM, $position) = split(/,/,$sortedteamsarray[$i]);
$wins[$i] = $brwins[$position];
}
# print(" wins = '@brwins'\n");
#### beginning of code to deal with scores against American League text
###
##
#
chomp($wlALstr); # get rid of final \n
@wlAL = split(/,/,$wlALstr); # create array of scores against American League teams
$ALwins = 0;
$ALlosses = 0;
foreach $t (@wlAL) {
($TEAM,$wl) = split(/:/, $t); # split team abbreviation with scores
($w,$l) = split(/-/, $wl); # split wins and losses
$ALwins += $w;
$ALlosses += $l;
# print (" '$t'\n");
}
# print ("AL wins = '$ALwins', AL losses = '$ALlosses'\n");
#
##
###
#### end of code to deal with scores against American League text
$wins[$numberofteams] = $ALwins."-".$ALlosses;
for ($i=0; $i<=$numberofteams; $i++) {
if ($i<$numberofteams && ($wins[$i] cmp "00")) {
$wins[$i] = $wins[$i]."-"; # separator between each team's wins and losses
}
# print("[$wins[$i]]");
}
# print ("\n");
# print(" win/loss against American League = '$wlALstr'\n");
# print $_;
$teamshash{$team} = [@wins]; # I need to put square brackets around @wins to get it to store the array in the hash
# this for loop does the same thing: for ($i=0; $i<=16; $i++) {$teamshash{$team}[$i] = $wins[$i];}
# for ($i=0; $i<=$numberofteams; $i++) {
# $teamshash{$team}[$i] = $wins[$i];
# }
# print ("\n\n");
}
# print " --> ";
# print $_;
}
#### beginning of code to create final row of scores of American League against these teams
###
##
#
$sortedlistindex = 0;
foreach $key (sort(keys(%teamshash))) { # this foreach loop creates the final row of wikipedia table
($ALwins, $ALlosses) = split(/-/, $teamshash{$key}[$numberofteams]); # access final array element of each team's array
# $teamshash{"_AL"}[$sortedlistindex] = $ALlosses."-".$ALwins;
$teamshash{"_AL"}[$sortedlistindex] = $ALlosses."-"; # the following foreach loop fills in the $ALwins so don't do that here
$sortedlistindex++;
}
$teamshash{"_AL"}[$numberofteams] = "%-%";
#
##
###
#### end of code to create final row of scores of American League against these teams
$sortedlistindex = 0;
foreach $key (sort(keys(%teamshash))) {
print "$key,";
for ($i=0; $i<=$numberofteams; $i++) {
($team, $position) = split(/,/,$sortedteamsarray[$i]); # get team portion of sorted teams array;
# $team now contains ($i)th team name,
# based on sorted list of team names
($n1, $n2) = split(/-/, $teamshash{$team}[$sortedlistindex]); # get first number in array entry
# this avoids having to create a
# separate hash of arrays
$teamshash{$key}[$i] = $teamshash{$key}[$i].$n1; # this adds losses for this team ($key) against each team
if ( !($teamshash{$key}[$i] cmp "%-%") ) {
$teamshash{$key}[$i] = "--"; # change entries of grid where team row intersects team column with "--"
}
if ($i==$numberofteams) {
print "$teamshash{$key}[$i]";
} else {
print "$teamshash{$key}[$i],";
}
}
# foreach $win ($teamshash{$key}) {
# print "[".$win."]";
# }
print "\n";
$sortedlistindex++;
}
# printf "\nversion is v%vd\n", $^V; # Perl's version
Ok, now below is the Perl program readtemplate.pl, which is used to read the output from the above Perl program, CnvBRtbl2ntrmd.pl. But this Perl program doesn't just have one input file, as with CnvBRtbl2ntrmd.pl, but 2 input files. The first input file is, as already stated, the output from CnvBRtbl2ntrmd.pl. The 2nd input file is the wikiscript for this wiki table we want to modify called 2010 NL Record vs. opponents. There is more explanation within this Perl file, which follows (note that I did not update this file for this year, so when it refers to 2009, just make that 2010 where appropriate):
readtemplate.pl
# This PERL program was originally completed on September 24, 2008. The way
# you use this is explained below:
# Let's say this file you are reading is called readtemplate.pl
# Let's also say you have a file which is the output from CnvBRtbl2ntrmd.pl
# called IFOFCC.txt which is the new table data gotten from the table called
# "Head to Head Records" on the web page
# http://www.baseball-reference.com/leagues/NL/2009-standings.shtml
# And let's say you have the wikiscript that describes the current version of
# the "2009 NL Record vs. opponents" table in WF2009NRVO.txt
# Then to get the wikiscript with the new table data you do the following at
# a command prompt:
# perl readtemplate.pl IFOFCC.txt WF2009NRVO.txt > new_wikiscript.txt
# Once you have done this, the new wikiscript to describe the updated table
# will be in the file new_wikiscript.txt. You can then paste that into the
# appropriate place in wikipedia where you are instructed to save the modified
# version of the wikiscript describing the "2009 NL Record vs. opponents" table
#
# Please note that for this perl program to work, there are certain assumtions
# which if no longer valid will cause this perl program to fail. Assumptions
# such as, the lines of the wikiscript describing the table are in the following
# form:
# | align=left | [[2009 New York Mets season|NY Mets]] || 0–0 || 3–2 || 0–0 || 2–1 || 0–0 || 4–5 || 0–0 || 0–3 || 2–1 || – || 3–1 || 3–3 || 1–2 || 3–1 || 0–3 || 6–1 || 2–1
# Note that all the data for each team is on one line, the win-losses are in
# this form (number followed by separator (- or –) followed by number) and
# separated by ||'s.
# This program also assumes the intermediary file being used as the first
# argument looks like this:
#
# ARI,--,3-3,2-1,0-3,3-3,3-1,0-0,2-6,2-2,0-0,0-0,0-0,3-3,2-4,1-2,1-2,2-1
# ATL,3-3,--,1-1,2-1,2-2,2-3,1-2,0-0,0-1,2-3,4-2,1-2,0-0,0-3,1-2,4-2,3-0
# CHC,1-2,1-1,--,2-2,1-1,3-1,5-2,2-2,3-3,0-0,0-0,2-1,3-3,1-1,3-6,0-0,0-0
# CIN,3-0,1-2,2-2,--,0-0,1-1,7-3,0-0,3-5,1-2,1-2,3-2,0-3,0-0,4-3,0-0,2-1
# COL,3-3,2-2,1-1,0-0,--,1-2,2-5,1-8,0-0,0-0,1-2,1-2,5-3,2-3,1-0,0-0,2-1
# FLA,1-3,3-2,1-3,1-1,2-1,--,0-0,1-2,3-4,5-4,2-4,0-3,0-0,0-1,0-0,6-0,1-2
# HOU,0-0,2-1,2-5,3-7,5-2,0-0,--,2-1,2-4,0-0,0-0,5-2,3-0,0-0,0-3,0-1,0-3
# LAD,6-2,0-0,2-2,0-0,8-1,2-1,1-2,--,0-0,3-0,3-2,0-0,6-2,5-4,0-0,1-1,1-2
# MIL,2-2,1-0,3-3,5-3,0-0,4-3,4-2,0-0,--,1-2,2-1,5-0,0-0,1-2,4-2,0-0,0-3
# NYM,0-0,3-2,0-0,2-1,0-0,4-5,0-0,0-3,2-1,--,3-1,3-3,1-2,3-1,0-3,6-1,2-1
# PHI,0-0,2-4,0-0,2-1,2-1,4-2,0-0,2-3,1-2,1-3,--,0-0,4-2,0-0,2-0,10-2,2-1
# PIT,0-0,2-1,1-2,2-3,2-1,3-0,2-5,0-0,0-5,3-3,0-0,--,2-1,0-0,4-5,3-1,1-2
# SDP,3-3,0-0,3-3,3-0,3-5,0-0,0-3,2-6,0-0,2-1,2-4,1-2,--,6-2,0-0,0-0,0-0
# SFG,4-2,3-0,1-1,0-0,3-2,1-0,0-0,4-5,2-1,1-3,0-0,0-0,2-6,--,2-1,4-2,1-2
# STL,2-1,2-1,6-3,3-4,0-1,0-0,3-0,0-0,2-4,3-0,0-2,5-4,0-0,1-2,--,2-1,2-1
# WSN,2-1,2-4,0-0,0-0,0-0,0-6,1-0,1-1,0-0,1-6,2-10,1-3,0-0,2-4,1-2,--,1-2
# _AL,1-2,0-3,0-0,1-2,1-2,2-1,3-0,2-1,3-0,1-2,1-2,2-1,0-0,2-1,1-2,2-1,--
#
# Notice that the final line starts with "_AL".
#
# Also note that this perl program does NOT modify the line that looks like
# this:
# As of ''May 24, 2009''.
# so you will still need to manually change that one line to correctly reflect
# the last day for which the new wikitable is valid. I have found that the
# people who update the baseball-reference table usually have on the bottom
# of the relevant web page that the statistics on that page have been updated
# as of that morning, so you will want to record the previous day as the
# correct "As of" date.
# use strict;
# use warnings;
while (<>) { # read in output from CnvBRtbl2ntrmd.pl
chomp($_);
push(@WL, [split(/,/,$_)]); # create 2-dimensional array @WL for storing all the win-loss data
if (/_AL/) { # I am assuming last line of this file starts with "_AL"
last;
}
}
$numteamsplusone = @WL;
for ($team=0; $team<@WL; $team++) {
for ($i=0; $i<@{$WL[$team]}; $i++) {
if (!($WL[$team][$i] cmp "--")) {
$WL[$team][$i] = "–"
}
}
}
if (1==0) { # this is just for easy debugging, normally line should read "if (1==0) {"
$numteamsplusone = @WL;
print "Number of teams + 1 = $numteamsplusone\n";
for ($team=0; $team<$numteamsplusone; $team++) {
$lenoflist = @{$WL[$team]};
print("lenoflist = $lenoflist\n");
for ($i=0; $i<$lenoflist; $i++) {
print("\[$WL[$team][$i]\]");
}
print "\n";
}
}
# $linenum = 1;
# while (<>) {
# print("$linenum. $_");
# $linenum++;
# }
# $linenum = 1;
# while (<>) {
# if (/( *\|\| *\d+[-–]\d+){8,}/) {
# print("$linenum. $_");
# $linenum++;
# }
# }
if (1==1) {
$team = 0;
while (<>) { # as each line of the original wikiscript for the "2009 NL Record vs. opponents" table is read in
if (/( *\|\| *\d+[-–]\d+){8,}/) {
# if this is a line that matches the following pattern:
# 8 or more of the following pattern:
# zero or more spaces followed by 2 vertical bars ("||") followed by
# zero or more spaces followed by
# one or more digits followed by a '-' or '–' followed by one or more digits
# typical matching line = "| align=left | [[2009 Philadelphia Phillies season|Philadelphia]] || 0-0 || 2-4 || 0-0 || 2-1 || 2-1 || 4-2 || 0-0 || 2-2 || 1-2 || 1-3 || – || 0-0 || 4-2 || 0-0 || 2-0 || 10-2 || 2-1"
s/\d+([-–])\d+/%s$1%s/g; # for this line, substitute any occurances of
# digit(s)-digit(s) (eg., "10-3") with "%s-%s"
# but keeping the same type of separator between
# wins and losses (- or –)
s/–/%s/g; # substitute any occurance of "–" with "%s"
# altered line = "| align=left | [[2009 Philadelphia Phillies season|Philadelphia]] || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s || %s–%s"
# print($_);
@a = @{$WL[$team]}; # transfer all elements of row $team of 2D array @WL into array @a
# the following for loop transfers from array @a, which holds a win-loss pair of
# numbers for each elements of the array (except for the one "–"), to another
# array @b which instead holds every number (win or loss) in a separate element
$bi = 1; # start index into @b array at 1
for ($ai=1; $ai<=@WL; $ai++) {
if ($a[$ai] =~ /[-–]/) {
@winloss = split(/[-–]/,$a[$ai]); # after split(/[-–]/,"12-5"); $winloss[0]=12 and $winloss[1]=5
$b[$bi] = $winloss[0];
++$bi;
$b[$bi] = $winloss[1];
++$bi;
} else {
$b[$bi] = $a[$ai];
++$bi;
}
}
$b[0] = $_; # A printf() type format string has been set up in $b[0] where every
# place you find a "%s", the value of the next element of the array @b
# will be substituted into that location when it is 'printf'ed out.
# For example, printf("Score is %s", "7-3") will output "Score is 7-3"
# but in PERL, you can put those 2 arguments in an array, so you can do
# $b[0]="Score is %s"; $b[1]="7-3";
# and then executing printf @b will output the same thing as before.
printf @b;
$team++;
} else {
print($_);
}
}
}
# Well, as of September 24, 2008, this thing does seem to work!
#
# June 5, 2009 - discovered that separator between wins and losses sometimes
# is regular hyphen '-' (\x2d) and sometimes is an en dash '–' (\x96). So I have
# now modified this file to take care of that. Also, I added code such that
# the separator types used in the wikiscript that is input as the second argument
# will be transferred over to output wikiscript. So whereever the last person who
# edited the wikiscript had used a regular hyphen ('-' (\x2d)) will still be there
# in the resultant wikiscript, and whereever the last person who edited the
# wikiscript had used an en dash ('–' (\x96)) this will still be there in
# the resultant wikiscript.
Please see http://en.wikipedia.org/wiki/User:Nejd/Scripts for a more up-to-date script for readtemplate.pl
Now, I created these myself because I had done some stuff in Perl and this seemed a perfect use for that language. When I see a problem like this (in this case, keeping this table up to date), and know that this is exactly the type of task that a computer was designed to accomplish, I believe you should have the option of using said computer to accomplish such a tedious task. Of course, however, this method might lead to other problems, which you have to watch out for. For example, as I was working on these programs, I noticed that people would keep changing what this wiki table looks like and what kind of characters they used. As soon as you write a program to read something on a computer, you have to watch out that the format of the data you are working off of doesn't change. But I started working on this in 2008. Then in 2009 I noticed it no longer worked, so I reworked it to take care of changes that had been made to both the source text table at BASEBALL REFERENCE and the table on Wikipedia. Now for 2010, the table in Wikipedia has been modified and modified, so I was really afraid these programs had broken (at least the 2nd one). But I tried it and it still seems to work, which shows that I made it pretty robust. Anyway, I created these but I am not really interested myself in using them to keep this table up to date. So what I am really hoping is that, someone who is really a big baseball person who really wants to keep this table current can come and take these files and use them to do that task as conveniently as possible. Once you understand how to do this it should take like a few minutes every time you want to update the table. And seeing as now they have a link reference to the source for the table's data, I hope when people use this method they will give credit to BASEBALL REFERENCE.