SABR is planning on revising their booklet on how to do baseball research and
putting it on-line. One of the sections in the guide will be a discussion of
statistical databases and websites. I was asked to work on this section, but my
experience dealing with baseball databases (convert Pete Palmer's 1994 Total
Baseball CD to my own format) is probably not going to be very helpful. So I was
looking for some advice and this seemed like a good place to start.

I guess my first question is whether anyone on this list would like to write (or
point me to a FAQ or existing guide) a tutorial on how to use Baseball
Databank's data?

Also, are there other statistical resources on the internet we should be
recommending (this should include both free and pay sites)? Ones that come to
mind right away are baseball-reference's play index and Pat Doyle's professional
baseball players DB, but I'm sure there are several others.

Any and all help/advice is appreciated.
Thanks for your patience.
Tom Ruane

> I guess my first question is whether anyone on this list would like to
> write (or point me to a FAQ or existing guide) a tutorial on how to use
> Baseball Databank's data?
>

There are a few out there.  Here's one:
http://statspeak.net/2008/11/building-a-sabermetricians-workbench-part-i.html

And you can quote Adler's book as well.

There's also the Retrosheet database:
http://tangotiger.net/wiki/index.php?title=Retrosheet_Database

And there are also tutorials around as to how to use that DB as well, if
you are going to be going there.

Finally, there's the PITCHf/x (Gameday) database and tutorial here:
http://www.beyondtheboxscore.com/2009/8/19/994666/saberizing-a-mac-4-pitch-f-x

I can think of at least a few more tutorials out there that I've seen in
the past year.

Tom

tjruane-san wrote:

> Also, are there other statistical resources on the internet we should be
> recommending (this should include both free and pay sites)? Ones that come
> to mind right away are baseball-reference's play index and Pat Doyle's
> professional baseball players DB, but I'm sure there are several others.

There are a couple of data sets available from NPB (Nippon
Professional Baseball) here:

     http://www.japanesebaseball.com/data/

The first one is mine.  The above link will allow you to download the
various tables as you want in CSV format, or you can download the
latest (I just updated it today) MySQL dump here:

     http://www.japanesebaseball.com/data/proyakyu.sql

The above are for past seasons.  I also handle in-season data for
Japan and Korea, but I'm not permitted to make those available to the
public yet.

---
Michael Westbay
System Administrator
JapaneseBaseball.com

Thanks for all the help. If the group doesn't mind, I might post a preliminary
draft to the File section in a few weeks for review.
Thanks again.
Tom Ruane

This could very well be a stupid question, but how similar is the data on
baseball-databank and baseball1.com? I thought that at one point these were the
same (although Sean provides the data in different formats), but have they
diverged since the inception of baseball-databank? If I compared the two
databases, how many statistical differences would I find?

Thanks.
Tom Ruane

Of the tables that intersects, I would have to say the data is identical.
I provided Lahman (and this group) with an Access script that loads all
the baseball-databank csv files directly into Access.  I will presume that
Sean did not change it.

Now, Forman also provides additional tables, like STATS/BIS xref IDs.
Whether Lahman also loads those (or others) into his database, I didn't
check.

In any case, Lahman can provide the final word if he's around.

Tom


> This could very well be a stupid question, but how similar is the data on
> baseball-databank and baseball1.com? I thought that at one point these
> were the same (although Sean provides the data in different formats), but
> have they diverged since the inception of baseball-databank? If I compared
> the two databases, how many statistical differences would I find?
>
> Thanks.
> Tom Ruane
>
>
>


---------------------------------------------
The Book--Playing The Percentages In Baseball
http://www.InsideTheBook.com

Tangotiger;
I found your "primpos.mdb" in the files section. I would like to use your
queries with the Lahman 5.6 database. My interest is career prime position
output. If I understand what you did, do I only need to copy  your career
related SQL queries (using 'playerID' instead 'LahmanID')?
Any assistance will be appreciated.
Mac

That would seem to be correct.  Try it out, and see what you get.

Tom

> Tangotiger;
> I found your "primpos.mdb" in the files section. I would like to use your
> queries with the Lahman 5.6 database. My interest is career prime position
> output. If I understand what you did, do I only need to copy  your career
> related SQL queries (using 'playerID' instead 'LahmanID')?
> Any assistance will be appreciated.
> Mac
>
>
>


---------------------------------------------
The Book--Playing The Percentages In Baseball
http://www.InsideTheBook.com

--- In baseball-databank@yahoogroups.com, "Tangotiger" <tom@...> wrote:
> In any case, Lahman can provide the final word if he's around.

Tom is correct. The underlying data is identical. The output formats are what's
different.

Sean Forman and I formed the baseball databank to help coordinate research and
avoid duplication of effort, as well as to avoid divergence and differences with
the underlying data.

The baseball databank primarily serves folks with some degree of dexterity for
working with relational databases.  While this probably represents the majority
of researchers, it's a smaller percentage of the overall users of the data. 
People without such skills are better able to use the data in MS Access or in
the flat file spreadsheets.  These also are used by third-party interfaces
including baseball sims.

But the root source for everything I distribute is the databank's SQL version.

Regards,
Sean Lahman

Tom;
Worked like a charm. Just what I needed.
Mac

--- In baseball-databank@yahoogroups.com, "Tangotiger" <tom@...> wrote:
>
> That would seem to be correct.  Try it out, and see what you get.
>
> Tom
>
> > Tangotiger;
> > I found your "primpos.mdb" in the files section. I would like to use your
> > queries with the Lahman 5.6 database. My interest is career prime position
> > output. If I understand what you did, do I only need to copy  your career
> > related SQL queries (using 'playerID' instead 'LahmanID')?
> > Any assistance will be appreciated.
> > Mac
> >
> >
> >
>
>
> ---------------------------------------------
> The Book--Playing The Percentages In Baseball
> http://www.InsideTheBook.com
>

It's in the Files section.  I don't remember if I put a Readme file with
it.  It's an Access query that figures out the primary position of every
player.  Download the file, and check it out...

Tom

> pardon my ignorance, but what is the primpos.mdb database?
>
> On Tue, Sep 1, 2009 at 10:21 PM, sgtmac5660@...
> <sgtmac5660@...>wrote:
>
>> Tom;
>> Worked like a charm. Just what I needed.
>> Mac
>>
>> --- In baseball-databank@yahoogroups.com, "Tangotiger" <tom@...> wrote:
>> >
>> > That would seem to be correct.  Try it out, and see what you get.
>> >
>> > Tom
>> >
>> > > Tangotiger;
>> > > I found your "primpos.mdb" in the files section. I would like to use
>> your
>> > > queries with the Lahman 5.6 database. My interest is career prime
>> position
>> > > output. If I understand what you did, do I only need to copy  your
>> career
>> > > related SQL queries (using 'playerID' instead 'LahmanID')?
>> > > Any assistance will be appreciated.
>> > > Mac
>> > >
>> > >
>> > >
>> >
>> >
>> > ---------------------------------------------
>> > The Book--Playing The Percentages In Baseball
>> > http://www.InsideTheBook.com
>> >
>>
>>
>>
>>
>> ------------------------------------
>>
>> http://www.baseball-databank.org/Yahoo! Groups Links
>>
>>
>>
>>
>


---------------------------------------------
The Book--Playing The Percentages In Baseball
http://www.InsideTheBook.com

Matthew;
If you feel that Tom's primpos.mdb is something you might be interested in
using, just be aware you need to make some changes to use it with later versions
of the Lahman Access database. Feel free to contact me if you need any
assistance.
Mac

--- In baseball-databank@yahoogroups.com, Matthew Gargano <mgargano@...> wrote:
>
> pardon my ignorance, but what is the primpos.mdb database?

For anyone interested I have uploaded a text file (MLB Outfielder Positions.txt)
to the files section. It contains around 480 major league outfielders and their
career OF position. I'm sure I missed one here or there but what do want from an
old geezer (VBG). The selections were based on the player's total games at that
position. No nitpicking allowed (VVBG).
Mac

Thanks. I actually wrote a few functions as part of a php library to
handle the data. One is a method of determining primary position and
another is a list of positions played by the player in at least a
variable percentage of their innings. I also have a determinant of SP
vs RP vs CL.  I have other functions as well such as
ReturnAgeAtSeasonStart, returnarrayofteamsplayedfor and a whole lot
more.


The only thing I want to add is a method of weighting more recent data
and possibly degrading data over a fixed/variable amount of years
(depending on age,position,etc.) for the data to make more accurate
determinations.

I think it's better to make a determination "on the fly"  as opposed
to using a fixed database especially if data is being updated at some
point.  A static database of player-position can get stale, especially
if the user forgets to include this years and used 2007's by accident.

I'm definitely interested in discussing further, especially if I
missed the point entirely - which is something I tend to do since I
try to think abstractly.

Matthew Gargano



On Sep 3, 2009, at 5:31 AM, "sgtmac5660@..." <sgtmac5660@...>
wrote:

> Matthew;
> If you feel that Tom's primpos.mdb is something you might be
> interested in using, just be aware you need to make some changes to
> use it with later versions of the Lahman Access database. Feel free
> to contact me if you need any assistance.
> Mac
>
> --- In baseball-databank@yahoogroups.com, Matthew Gargano
> <mgargano@...> wrote:
>>
>> pardon my ignorance, but what is the primpos.mdb database?
>
>
>
>
> ------------------------------------
>
> http://www.baseball-databank.org/Yahoo! Groups Links
>
>
>

PrimPos.mdb is dynamic.  It'll automatically recalculate the positions any
data the data set changes.

Tom

> Thanks. I actually wrote a few functions as part of a php library to
> handle the data. One is a method of determining primary position and
> another is a list of positions played by the player in at least a
> variable percentage of their innings. I also have a determinant of SP
> vs RP vs CL.  I have other functions as well such as
> ReturnAgeAtSeasonStart, returnarrayofteamsplayedfor and a whole lot
> more.
>
>
> The only thing I want to add is a method of weighting more recent data
> and possibly degrading data over a fixed/variable amount of years
> (depending on age,position,etc.) for the data to make more accurate
> determinations.
>
> I think it's better to make a determination "on the fly"  as opposed
> to using a fixed database especially if data is being updated at some
> point.  A static database of player-position can get stale, especially
> if the user forgets to include this years and used 2007's by accident.
>
> I'm definitely interested in discussing further, especially if I
> missed the point entirely - which is something I tend to do since I
> try to think abstractly.
>
> Matthew Gargano
>
>
>
> On Sep 3, 2009, at 5:31 AM, "sgtmac5660@..." <sgtmac5660@...>
> wrote:
>
>> Matthew;
>> If you feel that Tom's primpos.mdb is something you might be
>> interested in using, just be aware you need to make some changes to
>> use it with later versions of the Lahman Access database. Feel free
>> to contact me if you need any assistance.
>> Mac
>>
>> --- In baseball-databank@yahoogroups.com, Matthew Gargano
>> <mgargano@...> wrote:
>>>
>>> pardon my ignorance, but what is the primpos.mdb database?
>>
>>
>>
>>
>> ------------------------------------
>>
>> http://www.baseball-databank.org/Yahoo! Groups Links
>>
>>
>>
>


---------------------------------------------
The Book--Playing The Percentages In Baseball
http://www.InsideTheBook.com

> That's awesome Tom, I didn't know that, sorry for making an assumption.
> Is
> there anything that automatically updates data within the current season?
>

Best bet is to join the RetroSQL group.  There's a couple of (external)
ongoing things that are highlighted there.

Tom

On Wed, 26 Aug 2009 19:57:54 -0000, tjruane wrote:
[snip]
> Also, are there other statistical resources on the internet we should
> be recommending (this should include both free and pay sites)? Ones
> that come to mind right away are baseball-reference's play index and
> Pat Doyle's professional baseball players DB, but I'm sure there are
> several others.
[snip]

Tom:

The Baseball Cube *is* willing to sell data.  Never tried it, but it looks like
they deliver in a spreadsheet.

See this page:

http://thebaseballcube.com/help/data_Store.shtml

joel

> On Wed, 26 Aug 2009 19:57:54 -0000, tjruane wrote:
> [snip]
>> Also, are there other statistical resources on the internet we should
>> be recommending (this should include both free and pay sites)? Ones
>> that come to mind right away are baseball-reference's play index and
>> Pat Doyle's professional baseball players DB, but I'm sure there are
>> several others.
> [snip]
>

I have a fairly exhaustive list here:
http://tangotiger.net/wiki/index.php?title=Websites#Data

Tom

--- In baseball-databank@yahoogroups.com, "sgtmac5660@..." <sgtmac5660@...>
wrote:
>
> For anyone interested I have uploaded a text file (MLB Outfielder
Positions.txt) to the files section. It contains around 480 major league
outfielders and their career OF position. I'm sure I missed one here or there
but what do want from an old geezer (VBG). The selections were based on the
player's total games at that position. No nitpicking allowed (VVBG).
> Mac
>
I know you said no nitpicking allowed, but I should point out that these guys
have duplicate records:

davisch01
bondsbo01
winfida01
beltrca01
davisto02
guerrvl01
raineti01
oneilpa01
camermi01

THANKS,
KJOK

As I mentioned earlier, SABR is planning on updating its 2000 booklet, "How to
Baseball Research" and making it available on-line. An early version of the
chapter on statistical databases and websites is now available at:

http://www.retrosheet.org/Research/RuaneT/htdbr_stat_internet_chap.html

Please let me know of any corrections or suggestion for additions.

Thanks for your patience.
Tom Ruane

I joined and searched retrosql and did not see anything for current
year stats. Is there some kind of guide or readme on how things work
on the group?

On Thursday, September 3, 2009, Tangotiger <tom@...> wrote:
>
>> That's awesome Tom, I didn't know that, sorry for making an assumption.
>> Is
>> there anything that automatically updates data within the current season?
>>
>
> Best bet is to join the RetroSQL group.  There's a couple of (external)
> ongoing things that are highlighted there.
>
> Tom
>
>
>
> ------------------------------------
>
> http://www.baseball-databank.org/Yahoo! Groups Links
>
>
>
>

Judson Green wrote:

> Baseball Reference has had the SABR Home Run Encyclopedia for
> quite some time. It was announced by Sean on May 19, 2009:
>
> http://www.baseball-reference.com/blog/archives/1632
>
> and can be accessed by clicking the "more [+]" tab, then
> scrolling to "home runs" and clicking there. Alternatively, if
> you search a player from the home page, you'll see a tab that
> says "HR Log" below where it says "Standard Batting".

Thanks for the update. I have changed the linked chapter accordingly.

Tom Ruane