DRF Takes Down Servers

[Snag]

Can you believe it? Today, Labor Day, DRF takes their PP Download servers offline for "routine maintenance". When I called I was told to try back at a later date. They did not know how long they would be down.

I guess I shouldn't be surprised but I've paid in advance.

[SignUpKing]

Ridiculous that such maintenance is not done between midnight and 6 AM. Inexcusable!

[BillW]

They don't do routine maintenance on a holiday. They were obviously having problems.

[dutchboy]

The problems started saturday. If you had clicked several articles on the home page server error messages appeared. I assumed the problem was on my end.

[Dave Schwartz]

They seem to be up now, although I did not try ALL there services.

Ever notice how things usually fail at a bad time?


Dave

[Tom]

They must be run by the same people at You Bet who habitually fail on big days! :lol:

[speedking]

They were having problems with the server since Friday night. Data downloads hung and froze constantly every day. Tried different machines with same results. Ended up downloading 1 track at a time and it took about an hour, maybe more, for all tracks:mad: Each Day!

Thought it was my DSL service at first since nobody else posted a problem and I jumped all over some tech guy in Bangladesh or India:blush:

speedking

[richrosa]

This might not have been totally their fault. There's a slight chance from the descriptions above that it could have been a denial of service attack. Most modern gateways can fend off such an event, but maybe their equipment is older.

[Snag]

This might not have been totally their fault. There's a slight chance from the descriptions above that it could have been a denial of service attack. Most modern gateways can fend off such an event, but maybe their equipment is older.

I called and talked to them at length. In their own words it was for routine maintenance. I was told they took their servers down themselves. This isn't the first time I've heard this explanation. It is the first time I've seen it on a holiday.

IMHO, it's another example of an attitude towards their paying customers.

[njcurveball]

In their own words it was for routine maintenance.

Anyone in Management who authorized this should be given their pink slip. :ThmbDown:

Mark Twain comes to mind here.

It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt.

[richrosa]

Marc Attenberg is in charge of DRF's websites. He's a real good guy and posts here from time to time (Marc @ DRF).

I'm certain that he will explain this event. I can tell you first hand that Marc cares about the DRF customer.

[rrbauer]

Marc Attenberg is in charge of DRF's websites. He's a real good guy and posts here from time to time (Marc @ DRF).

I'm certain that he will explain this event. I can tell you first hand that Marc cares about the DRF customer.

The "techies" do not work for Marc. Indeed, they do not work for anyone!!!

[thespaah]

Can you believe it? Today, Labor Day, DRF takes their PP Download servers offline for "routine maintenance". When I called I was told to try back at a later date. They did not know how long they would be down.

I guess I shouldn't be surprised but I've paid in advance. I called customer service after they charged my card for the SAR PP's and they refunded my money. I was real PO'ed that they would do what they did..Well gues what happened/..The PP's loaded anyway. Albeit very slowly..So slowly that I made the purchase at 12.45 and the PP's finally appeared at 1.25 pm..

[richrosa]

The "techies" do not work for Marc. Indeed, they do not work for anyone!!!

What do you mean?

[speedking]

I called customer service after they charged my card for the SAR PP's and they refunded my money. I was real PO'ed that they would do what they did..Well gues what happened/..The PP's loaded anyway. Albeit very slowly..So slowly that I made the purchase at 12.45 and the PP's finally appeared at 1.25 pm..

When I tried to download more than 1 file at a time the application timed out and I had to start over. 1 file at a time went through, but slowly.

speedking

[rrbauer]

What do you mean?

A little slow today?

[midnight]

Being that we're not mindreaders, perhaps you can explain to those of us who aren't able to decipher what you mean.

[Ron]

I called customer service after they charged my card for the SAR PP's and they refunded my money. I was real PO'ed that they would do what they did..Well gues what happened/..The PP's loaded anyway. Albeit very slowly..So slowly that I made the purchase at 12.45 and the PP's finally appeared at 1.25 pm..

You were guilty of waiting until the last minute! ;)

[Gibbon]

This might not have been totally their fault. There's a slight chance from the descriptions above that it could have been a denial of service attack. Most modern gateways can fend off such an event, but maybe their equipment is older. All enterprise level gateways within the past 3/4 years. But this only obfuscates the main point. This does not happen to Equibase. Last year HDW was struck by lighting and up within hours.

If one was cynical, over the years taken in panoply, DFR does seem yet another company overly concerned with the bottom line. Is it any wonder Moss introduced his pace figs only after every other data provider had pace figs?

DRF represents the old guard. Modern conveniences has relegated the DRF irrelevant.







__________________________
Lottery: A tax on people who are bad at math.

[Marc At DRF]

This was not routine maintenance. This was server issues that were not fully resolved until yesterday afternoon. Our back-end is all hosted by The Jockey Club's technology group, and we haven't had problems like this in quite a while. That said, it's inexcusable. On a customer service front, somebody in Chicago was giving out bad information. We would never plan "routine maintenance" on Labor Day weekend, ugh.

Apologies to all, there's really no good excuse for it. If you weren't able to get something that you paid for, please contact Cservice2@drf.com and they'll set you up either with refunds or credits to your account. Tell them I sent you and tell them I said they should be generous with you!

As an aside, I was on the site every weekend day and never had problems and didn't even hear about it until yesterday afternoon. I wish I would have heard about it earlier, as it was fixed within a couple hours of when I first found out.

[xtb]

Marc, do you know if/when DRF will have Formulator PP's for Presque Isle Downs?

[NoDayJob]

Can you believe it? Today, Labor Day, DRF takes their PP Download servers offline for "routine maintenance". When I called I was told to try back at a later date. They did not know how long they would be down.

I guess I shouldn't be surprised but I've paid in advance.

Is the DRF building their own confuser servers or are they made in China by rickshaw drivers and agrarian peasants? Perhaps the servers are O.K., but their software may leave the DRF a couple of days late and many dollars short. I'm glad I still use Post Time Daily for my software. :D

[Snag]

This was not routine maintenance. This was server issues that were not fully resolved until yesterday afternoon. Our back-end is all hosted by The Jockey Club's technology group, and we haven't had problems like this in quite a while. That said, it's inexcusable. On a customer service front, somebody in Chicago was giving out bad information. We would never plan "routine maintenance" on Labor Day weekend, ugh.

Apologies to all, there's really no good excuse for it. If you weren't able to get something that you paid for, please contact Cservice2@drf.com and they'll set you up either with refunds or credits to your account. Tell them I sent you and tell them I said they should be generous with you!

As an aside, I was on the site every weekend day and never had problems and didn't even hear about it until yesterday afternoon. I wish I would have heard about it earlier, as it was fixed within a couple hours of when I first found out.

Thanks Marc, but your folks on the phone were doing more than giving out bad information. The recording says the calls may be recorded. Someone may want to listen to some of them.

Ok, it's over for now. I would hope that plans are in place to provide us with backup servers in the future.

[turfnsport]

I don't know how long the servers were down, it affected me for about 45 mins while using Formulator...Over the past 7 or 8 years, however long I have used DRF.com, I can't recall any other time I could not get PP's...

one time this decade...I can live with that.

MikeD

[King Ritchie]

Marc states "Our back-end is all hosted by The Jockey Club's technology group"

What does this mean?

[richrosa]

Jockey Club Information Services is the outsourcing provider for DRF's website operations. They provide the database, website, and hosting support to run DRF.com.

[King Ritchie]

Jockey Club Information Services is the outsourcing provider for DRF's website operations. They provide the database, website, and hosting support to run DRF.com.

When 911 hit NY City, as I remember DRF was down for a month or so. So if we have another 911 in NY City then the DRF racing database is now safe? If so, that's good to know.

[Marc At DRF]

-Yes, we're much better set-up now in case of a big emergency like a terrorist attack or power outage or whatever.

-Regarding the server issue, it was a third-party software tool that keeps the site's servers in synch. This tool needs to be replaced, and it will be.

-Formulator for PID is quite literally a few hours of work to implement, but the developer doesn't really work for us regularly anymore, as we're in the process of migrating formulator to a web-based version later this year. Accordingly, we're waiting for him to make the fix--I thought it would have been last week but instead, we haven't heard from him. So hopefully this week. It speaks to the vulnerability of both the software and to having an outsider develop it for us over the years, even if the software has proven to be a great tool.

[Ron]

When 911 hit NY City, as I remember DRF was down for a month or so. So if we have another 911 in NY City then the DRF racing database is now safe? If so, that's good to know.

"911" hit more than NYC and if something that big happens again, I can do without DRF past performances for a month.

[DJofSD]

-Regarding the server issue, it was a third-party software tool that keeps the site's servers in synch. This tool needs to be replaced, and it will be.

I work in the IT industry. I've been following the thread and considered it just another example of how things can go bad, impacting the "customer facing" part but never really any details of what really went wrong. This is a close as it comes to "naming names, etc." without actually being on the inside doing the post mortem. Good show and thanks for the additional details.

[Gibbon]

...past 7 or 8 years, however long I have used DRF.com, I can't recall any other time I could not get PP's...one time this decade...I can live with that. I would wager a guess you're a casual race downloader.



... made in China by rickshaw drivers and agrarian peasants.....

...911 hit NY City...DRF was down for a month.......

....folks on the phone were doing more than giving out bad information.......

Some may draw the erroneous conclusion this is DRF bashing. Trust me it is anything but. Rather, a cry for help based on past extreme frustrations dealing with DRF. This observer has come to conclude dealing with DRF is equivalent to dealings with our Fed. Govt. Wait for a bridge collapse then send a team to study reasons why. Write a bureaucratic report – file said report in some secret bucker so no one can read it. Until next bridge collapse; later, place blame on previous owners.

With so many choices of data providers and news sites such as bloodhorse.com, equidaily.com and thoroughbredtimes.com {breaking news here (http://www.thoroughbredtimes.com/national-news/2007/September/04/nyra-retains-racing-franchise.aspx) } DRF is relegated to irrelevant status. All this and as of yet only limited support for third party apps.


"911" hit more than NYC and if something that big happens again, I can do without DRF past performances for a month. Both “sheets” providers are located in downtown Manhattan. Thorograph was up in a day and a half.

[turfnsport]

I would wager a guess you're a casual race downloader.


If you consider downloading 4-5 tracks a day, 6 days a week and using Formulator software 6-7 hours a day, six days a week casual, then yes.

MikeD

[northerndancer]

I left DRF a long time ago and have never been bothered by that decision.

I chose to go to Handicappers Data Warehouse. They have a much better site with much more useful information.

Actually I credit HDW for making me a much better punter.

Here is a link to the site http://www.horsedata.com/

[xtb]

-Formulator for PID is quite literally a few hours of work to implement, but the developer doesn't really work for us regularly anymore, as we're in the process of migrating formulator to a web-based version later this year. Accordingly, we're waiting for him to make the fix--I thought it would have been last week but instead, we haven't heard from him. So hopefully this week. It speaks to the vulnerability of both the software and to having an outsider develop it for us over the years, even if the software has proven to be a great tool.

Thanks Marc. While the programmer is adding PID, can you have him add Zia Park back? I think it was inadvertantly left out with the last upgrade of Formulator.

[Marc At DRF]

Yeah, we're going to try to get him to do Zia too.
It's awkward right now, as we've effectively ceased development on the software version of Formulator, but the game didn't cease developing along with us. We're excited about web Formulator, but stuff like this is certainly a hassle. Update: The old developer was traveling and will fix this stuff tomorrow.

[Gibbon]

"....game didn't cease developing along with us. We're excited about web Formulator...."

Why not allow serious user to Beta test? Letting the customer IN will lead to more productivity. Open source handicapping, just think of the possibles. O' yes I'm serious.

DATA is where the real profits are made.

[Snag]

-Yes, we're much better set-up now in case of a big emergency like a terrorist attack or power outage or whatever.

-Regarding the server issue, it was a third-party software tool that keeps the site's servers in synch. This tool needs to be replaced, and it will be.


Thanks for the additional information and follow-up Marc. I would hope that back-up servers are in the future. I know that my boss would be adding up the cost of lost revenue if this happened to us. I would hate to be the one telling him we lost x amount because of outdated software.

[DJofSD]

Thanks for the additional information and follow-up Marc. I would hope that back-up servers are in the future. I know that my boss would be adding up the cost of lost revenue if this happened to us. I would hate to be the one telling him we lost x amount because of outdated software.

Any publicly traded company has a fiduciary duty to provide for recovery of computer servers and functionality.

[Marc At DRF]

-We're not a publicly traded company and we have back-up servers. The errors thrown merely required a restart to clean up the mess; it took too long for us to realize the problem existed.

-I was wrong about Zia; it was fixed late last year, near the end of the meet.

-On beta testing Formulator, heck yes we need a whole lotta testers. This is a pretty enormous project. The look, feel, and functionality of the software needs to migrate to a web environment (in this case, Adobe Flex). At first, we're looking primarily to mimic current functionality, plus to clean up a few things that aren't that great about the existing functionality (lack of Moss Pace Figs as an option, for example). Then we're going to grow it...

If you're interested in beta testing, please e-mail me at mattenb@drf.com and I'll add you to the list...

[njcurveball]

-We're not a publicly traded company and we have back-up servers. The errors thrown merely required a restart to clean up the mess; it took too long for us to realize the problem existed.




MUCH respect to you Marc for giving a clear answer and accepting the responsibility. If you could just pass this attitude to the rest of the people involved, I think we all would be much happier with DRF.

[NoDayJob]

Yeah, we're going to try to get him to do Zia too.
It's awkward right now, as we've effectively ceased development on the software version of Formulator, but the game didn't cease developing along with us. We're excited about web Formulator, but stuff like this is certainly a hassle. Update: The old developer was traveling and will fix this stuff tomorrow.

Marc,

Here's a tip for the folks at DRF. Get an in house software developer. It'll cost more, but in the end DRF will regain the respect it once had. My software will use any data source that is available, so it really doesn't hurt me personally, but a lot of folks here depend on the DRF. MPO.

[King Ritchie]

Marc,

Here's a tip for the folks at DRF. Get an in house software developer. It'll cost more, but in the end DRF will regain the respect it once had. My software will use any data source that is available, so it really doesn't hurt me personally, but a lot of folks here depend on the DRF. MPO.

Maybe DRF can find someone on the dance floor this time.