MAIN FORUM

Comments

  • JustinAtFanduel

    Fanduel Rep

    Lineup display issue and resolution

    Hi all,

    First, our apologies in taking so long to post a response but we wanted to ensure we had all the facts and the issue was researched appropriately before putting out a statement.

    Here’s a brief update on last night’s issues. We have a team working on both aspects of this:
    1. Making sure that we settle contests in a way that none of condia’s opponents feel that they’ve been adversely effected by the contest results. This may take several more hours.
    2. Making sure that this can’t happen again.

    In the meantime, here’s a detailed technical explanation of what happened. Feel free to ask questions about it, but please understand that our focus right now is on getting the issue fixed so we may not get to answer follow up questions today.
    The way our system works is that we have a main database which holds all the entries and updates a user has made to their lineups. This is updated directly when a user makes changes, and is used by the settlement system to determine outcomes of the contest. We always take the content of this database as definitive. According to this database, condia made changes to all his teams comfortably before the start of the games.

    In order to be able to display all the pages on the site quickly to our volume of users we run several copies of this database – these are kept up to date from the main database using standard replication techniques. These generally reflect updates to the main database within a second, although they do occasionally run slightly later than that.

    Live scoring is a more complex issue – calculating and displaying positions in tables on a large volume of data within a couple of seconds of getting an update can’t really be done if you need to get the data from a database every time. What we do for this is store all the table information in memory on another system (something between a copy of the database and a cache). This is fed continuously with updates from one of the copy databases – basically the live scoring system hits up the database every few seconds, and gets all changes with a timestamp later than the timestamp of the last change it processed, plus a 10 second grace period before that to catch anything that happened out of order.

    Growth in the size of the database over the season reached a point over Christmas where the copy database that the live scoring system gets its data from is only just keeping up, and large changes like an export from condia can tip it over into being more than 20 seconds out of date for a minute or two. This appears to have led to condia’s updates not making it through to the live scoring system.

    Since before the game the rosters that you can see as a user are delivered from the database, condia would have had no reason to be concerned – all his changes were showing correctly. Once the games started however the lineup he was seeing changed back to the earlier lineup. When the technical team became aware of the issue some time later, they restarted the live scoring system, which causes it to do a complete reload of data from the database. At this point, all of condia’s line-ups changed to reflect the changes which had been made hours before and not picked up.

    We are making some changes today to stop this happening – firstly we are increasing the size of the database servers so they will be able to stay comfortably on top of the load, and secondly we are increasing the grace period, so if things do start to run slowly we will still pick up all changes.

  • Emoney21

    good call bats

  • stlcardinals84

    Leading RG Analyst

    • 347

      RG Overall Ranking

    • 2018 DraftKings FBWC Finalist

    • 2018 DraftKings FBBWC Finalist

    @waiverwire said…

    Condia plays in so many more contests than anyone else that almost any issue that’s related to contest volume happens in his account before anyone else’s.

    Are you guys essentially turning a blind eye to the need for a matchup blocker? I see it discussed all the time but haven’t really ever seen it addressed by a FD rep.

    I know condia plays a ton of volume, but he’s essentially dried up all posting of h2h games at $50 and up. There would be a gigantic increase in people posting these with a matchup blocker in place.

    There has to be a valid reason that pretty much every other site in the industry has already implemented this feature.

  • bluestem89

    • 2014 FanDuel WFBC Finalist

    • 2016 FanDuel MLB Playboy Mansion Finalist

    good read quality post imo

  • miggytoddbest

    • 2014 FanDuel WFBC Finalist

    FD response: Once the games started however the lineup he was seeing changed back to the earlier lineup. When the technical team became aware of the issue some time later, they restarted the live scoring system.

    Not Condia volume, but at times I have over 200 lineups going live. Never a “incorrect lineup”

    As I said last night, this explanation is 100% where people have to just take your word on it.

    There are many topics other people brought up, (gallinari for one) that you guys have blown by. I know investigating takes time, but 12 hours later you have really said nothing different than you did last night.

    You’ll get 50% to accept this.

  • Emoney21

    I would be able to post h2hs finally!

  • waiverwire

    @Natsfan said…

    Condia didn’t point it out? That would be shocking to me.

    The scores were close enough that he may not have realized exactly what happened with the payouts. Also he knew we were still investigating why his export ‘failed’ so he may have thought we knew.

  • bluestem89

    • 2014 FanDuel WFBC Finalist

    • 2016 FanDuel MLB Playboy Mansion Finalist

    agree with cardinals, every game i post gone within seconds, all good tho just gets old playing him everyday more likely to be a wash in the long run, i’ve also asked them about matchup blocker and they could care less, they love $ and dont care who they get it from and they know he has the biggest roll so y change it

  • Jup13

    @JustinAtFanduel said…

    We are making some changes today to stop this happening – firstly we are increasing the size of the database servers so they will be able to stay comfortably on top of the load, and secondly we are increasing the grace period, so if things do start to run slowly we will still pick up all changes.

    While the upgrade is good news, It’s a damn shame that with how obvious it was that the site needed work, it took something as drastic as this to push fanduel to finally do something.

  • stiles

    @waiverwire said…

    Not at all. I’m saying that if there’s a bug related to entering a ton of contests he’s the one who’s likely to run into it first.

    I find irony in your avatar, Waiverwire. The old man(Fanduel) built this “Matrix” and Neo(Condia)can pretty much do whatever he wants in said “Matrix”.

    It’s Funny.

  • sethayates

    @Emoney21 said…

    Why should I trust you?

    You don’t have to. If you don’t trust them don’t allow them to hold onto your money. Surely, you can’t be stupid enough to freely send money to a site and then think they are ripping you off yet send them more.

  • miggytoddbest

    • 2014 FanDuel WFBC Finalist

    @DevonD9 said…

    Obviously it seems, you’re one of those theorists and just signed up here to state an obnoxious opinion. I appreciate what Justin is doing other then some people. I don’t believe there is any conspiracies behind this.

    Never listen to anybody in FanDuel chat. Most of them just lie because they have nothing better to do most times. Unless you hear it from an actual admin.

    I highly doubt an admin actually told you Condia called them. So please leave all that drama and conspiracy out of here.

    Do you really have the right to call someone out? I don’t know this guy, but just to enlighten you, you are no more of an important person in this thread than I am or a guy that signed up yesterday.

    So, maybe a RG Moderator can address DEVOND9 calling some he disagrees with obnoxious, a drama queen and a conspiracy theorist. Consider this my email requesting deletion.

  • Natsfan

    @waiverwire said…

    The scores were close enough that he may not have realized exactly what happened with the payouts. Also he knew we were still investigating why his export ‘failed’ so he may have thought we knew.

    “The scores were close enough that he may not have realized exactly what happened with the payouts.”

    The scores were 12 points a part I believe that is substantial, Dinkpiece posted an example of Condia losing bc of it for one thing.

    “Also he knew we were still investigating why his export ‘failed’ so he may have thought we knew.”

    Are you saying your investigation didn’t uncover the fact that his export didn’t fail?

  • Jup13

    Also, I feel like Im beating a dead horse from another thread..but this “grace period” thing should have everyone thrown off. What is it? Fanduel has this notice clearly posted on the quick player replace page:

    ****Important: Contests will be updated one by one, so if you wait until the last minute before contest lock time, it is possible that only some of your lineups will be updated before time runs out.****

    Since Condia had TWO lineups friday, one with Gallo, one without, it would seem to me that THIS was the issue with THAT. However, yesterday’s problem was different than the gallo issue.

    The gallo issue made sense to me due to the lineups not all being the same, which would again, indicate that what the warning says, happened.

    Yesterday’s events have no plausible explanation from anything I have read so far.

  • hambazaza

    RG Blog Program Manager, 2014 RG Party Beer Pong Champion

    • Blogger of the Month

    • Beer Pong Champion

    what i’m understanding from this is:

    the settlement system uses the main database, which is updated quite regularly

    the main database has copied made into it, one of those gets used as the live scoring display database. the live scoring display database didn’t have the latest changes copied over in time for the games when they started. this caused the gallinari issue. the copies it seems are constantly being made of the main database.

    they weren’t aware of this issue apparently until it happened two nights in a row, and that led them to misdiagnose the symptom on the first night. and they eventually figured it out last night when they restarted the live scoring database.

    most people apparently don’t notice this because a 10-a few hundred lineups wont be affected. this caused the condia issue to stand out because he probably has thousands of lineups a day? I dont know what his actual volume is, but he’s in every single game that goes up all day!

    thats how i’m understanding it, and it makes sense to me. I’m not saying you should trust it, and im not saying you should believe it, but I am keeping my money at fd (all $200 of it, give or take) and if you really have a hard time with what happened or their explanation, think about this: the sheer volume of fd makes them the only ones who will experience an issue such as this, nobody else can probably come close to this issue as presently constructed. so if everybody migrated to site abc123 then eventually that site will also have the same issue.

  • waiverwire

    I realize that Justin’s initial post was pretty technical, but I think some of the more well thought out objections in here are misunderstanding the nature of the ‘grace period’. Assuming that I’m understanding things right, the problem could happen at any time. Not only in the seconds before games start.

    Also, the issue was with lineup export, not quick player replace.

  • sethayates

    @hambazaza said…

    what i’m understanding from this is:

    the settlement system uses the main database, which is updated quite regularly

    the main database has copied made into it, one of those gets used as the live scoring display database. the live scoring display database didn’t have the latest changes copied over in time for the games when they started. this caused the gallinari issue. the copies it seems are constantly being made of the main database.

    they weren’t aware of this issue apparently until it happened two nights in a row, and that led them to misdiagnose the symptom on the first night. and they eventually figured it out last night when they restarted the live scoring database.

    most people apparently don’t notice this because a 10-a few hundred lineups wont be affected. this caused the condia issue to stand out because he probably has thousands of lineups a day? I dont know what his actual volume is, but he’s in every single game that goes up all day!

    thats how i’m understanding it, and it makes sense to me. I’m not saying you should trust it, and im not saying you should believe it, but I am keeping my money at fd (all $200 of it, give or take) and if you really have a hard time with what happened or their explanation, think about this: the sheer volume of fd makes them the only ones who will experience an issue such as this, nobody else can probably come close to this issue as presently constructed. so if everybody migrated to site abc123 then eventually that site will also have the same issue.

    Shhhhh! I’m staying at FanDuel also but I was kind of hoping all the trolls would go somewhere else. When I was in the military I lived in base housing. There was a huge ant problem. One month they would come out and bomb my house and the ants got driven to my neighbors. The next month they bombed my neighbors and the ants came back. This back and for went on for several months. Hopefully the problem gets fixed but if not I’m ok with a month or so of peace as the ants go somewhere else.

  • merlik4

    Also chat banning people because you didn’t like what they were saying and they weren’t spamming it is in poor taste…

  • Emoney21

    I might still play FD

  • pokerrob1970

    DraftDay BLB Finalist

    Waiverwire: I know i am not the easiest guy to get along with but you have always been upfront and honest with me and others on RG. But no matter what you guys say or do outside of having a 3rd party audit is going to appease people and some not even with that. FD needs new software and has for a couple of years. I really don’t think the standard “we are on top of it” is going to fly this time around. GL

  • headChopper

    RG Contributor (OG Status)

    • 253

      RG Overall Ranking

    • Ranked #44

      RG Tiered Ranking

    • x2

      $1M Prize Winner

    • 2014 DraftKings FBWC Finalist

    @stlcardinals84 said…

    Are you guys essentially turning a blind eye to the need for a matchup blocker?

    I know condia plays a ton of volume, but he’s essentially dried up all posting of h2h games at $50 and up. There would be a gigantic increase in people posting these with a matchup blocker in place.

    This is my view also. Not sure what the hold up is. It seems like every other site has rolled out this feature pretty seamlessly recently. I think you’d be surprised that the drop off in H2H action isn’t that severe when you consider all the new lobby activity that would occur

  • Emoney21

    @headChopper said…

    This is my view also. Not sure what the hold up is. It seems like every other site has rolled out this feature pretty seamlessly recently. I think you’d be surprised that the drop off in H2H action isn’t that severe when you consider all the new lobby activity that would occur

    Hallelujah CHOP

  • killfast

    He probably just selects a random lineup and enters it into all of his contests so he can get registered. He then goes back and creates the lineup he wants and exports it.

  • mdube16

    • x2

      2014 FanDuel WFFC Finalist

    • 2018 FanDuel WFFC Finalist

    @headChopper said…

    This is my view also. Not sure what the hold up is. It seems like every other site has rolled out this feature pretty seamlessly recently. I think you’d be surprised that the drop off in H2H action isn’t that severe when you consider all the new lobby activity that would occur

    Another STRONG agree. I too have suggested this but clearly they arent interested. The “play a private game” answer isnt good enough for me.

    Mdube16

  • uncleslim

    I could not agree more. I can’t think of any situation in the world where one person is allowed to monopolize a situation that maximizes the companies bottom line. It would not only increase every player’s chances at profit and having fun with diverse match-ups but increase traffic which in turn increases the site’s bottom line.

  • troveur

    @Natsfan said…

    I’m definitely not a conspiracy theorist I’m just using logic to deduce opinions.

    As soon as you realized the Lineup was wrong why not post on RotoGrinders.com?

    Why wasn’t this addressed afters Fridays issue?

    Why did a FanDuel rep say that Lineups can’t be changed after they’re submitted when in fact they can be?

    Well I would say that Fanduel is not exactly the best DFS company out there man and expecting a quick common sense response to issue after issue is way too much to ask. Look how long they have dragged their feet on matchup blocker, fixing the awful chat and other necessary features.

    Biggest(not for long) and most definitely not the best.

  • X Unread Thread
  • X Thread with New Replies*
  • *Jumps to your first unread reply

Subforum Index

RotoGrinders.com is the home of the daily fantasy sports community. Our content, rankings, member blogs, promotions and forum discussion all cater to the players that like to create a new fantasy team every day of the week.

If you or someone you know has a gambling problem, crisis counseling and referral services can be accessed by calling 1-800-GAMBLER (1-800-426-2537) (IL). Gambling problem? Call 1-800-Gambler (NJ/WV/PA), 1-800-9-WITH-IT (IN), 1-800-522-4700 (CO) or 1-800-BETS OFF (IA). 21+. NJ/PA/WV/IN/IA/CO/IL only.