Jump to content
  • 0
Sign in to follow this  
Kanamori

Ptah - DKP site - Out of Memory Errors

Question

Ptah,

 

Been getting a decent number of out of memory errors while doing actions the DKP site the past 2 days.  I'm not seeing what might be causing such a huge memory draw, but something appears to be hogging some recently.  Can you take a look at it please.  Thanks!

 

Kana

Share this post


Link to post
Share on other sites

13 answers to this question

Recommended Posts

  • 0

It's related to the size of the raid table and the site running a cache renew on it. Everytime it runs, it uses up all of the memory the site has and reboots mysql to finish. Looking at when you were entering in the last set of raids, it looks like it ran a cache renew right as you were entering in the raids so that's why you saw it go out of memory.

I'll fire up the test dkp site and update it to the latest source and then try consolidating some raids again to see what happens. We may have to just write down the current dkp values of everyone, do a complete consolidation and then adjust everyone back to their current dkp values because I have a feeling the consolidation is going to eat raids in the process.

Share this post


Link to post
Share on other sites
  • 0

I'm ok with archiving the older raid data and consolidating it, especially stuff from 1-2 years ago heh.  Thanks for looking into it.  I was kinda along the same path as you there with it and figured I just happened along at a bad time to do some DKP work, but I'm definitely no veteran or expert at the IT side of database management.  I really do appreciate your work on this!

Share this post


Link to post
Share on other sites
  • 0

The consolidation tool uses events instead of raids so we'll need to create events for some of the events that have raids that are tied to them that are a year old or newer and move those raids over to the copy event.

I know all the old expansions are usually in the name of the events so we don't have to worry about them so I can knock those out fairly quickly.

I'll probably do this in stages just to see how things go. If we need to roll back, I'll of course back everything up before I even start.

Share this post


Link to post
Share on other sites
  • 0

Ya, outside of maybe original and kunark, those should have been made with the expansion as the first part of the name.  I think I had gone back and renamed some older stuff to this manner near when I first took up the DKP stuff, so most things should be pretty easy.

Share this post


Link to post
Share on other sites
  • 0
On 8/21/2018 at 5:58 PM, Kanamori said:

I'm ok with archiving the older raid data and consolidating it, especially stuff from 1-2 years ago heh.  Thanks for looking into it.  I was kinda along the same path as you there with it and figured I just happened along at a bad time to do some DKP work, but I'm definitely no veteran or expert at the IT side of database management.  I really do appreciate your work on this!

I'll be working on this today. I forgot to mention with the consolidation tool, once we consolidate a particular event, it deletes all the raids up underneath it, no archiving. It adds up all the dkp points for people in those raids and adds an adjustment for it.  I'd like to delete characters who haven't raided in a year or more and then consolidate raids.

I went back a year and pulled up all the characters who haven't raided on and before a year ago. Let me know if you approve of this list.

https://docs.google.com/spreadsheets/d/1QsGfd1jmRjL_vcaYnFU6hSRKgWrTlVIjda6H_sPA1a8/edit?usp=sharing

  • Like 1

Share this post


Link to post
Share on other sites
  • 0

Overall, the idea looks good and the list looks ok.  Thanks for looking into and fixing this!  The out of memory errors was getting pretty bad again due to the shear amount of data we have stored.

 

I'd say going up to 6 months or so behind now and archiving it all is good.  That's at least 2 expansions in almost any case that I can think of and is past any point where I would reasonably make a correction.  Could we use this as a point to do a database cleanup periodically going forward?  If at all possible, getting a copy of the list/DKP alterations made/to be made and keeping as an off-site archive would be useful (like in an Excel table with name - DKP from consolidation) and posting those in the Raid Records Administration forum in case I need to make/recreate someone for any reason.

Share this post


Link to post
Share on other sites
  • 0

Ptah,

 

I noticed that I was missing 300 DKP when I was putting in DKP just now.  Not quite sure what would have caused that.

 

EDIT - Just looking at the logs, it appears you're still working on the database.  Didn't expect you up at this time still working.  Not sure if my missing DKP is part of your script and the adjustment for the deleted raids is done later or not, so there might not be an issue.  Not a lot of us from back then are still part of the guild heh.  My bad.

 

I'll hold of posting DKP until after I wake up this morning(-ish)

Share this post


Link to post
Share on other sites
  • 0

I'm done for the time being. Classic, Kunark, and Velious raids are all removed along with events too. I'll go through and add adjustments to everyone where their dkp values changed.

Share this post


Link to post
Share on other sites
  • 0

Going to wait a half hour and then check again. It's still adjusting things behind the scenes since I've seen dkp values change.

Share this post


Link to post
Share on other sites
  • 0

I removed the raid you added since it conflicts with the records I took. Once I'm done with adjustments I'll let ya know so you can add raids.

Share this post


Link to post
Share on other sites
  • 0

Ok done with adjustments. You can enter raids now.

Share this post


Link to post
Share on other sites
  • 0

Site performance improved drastically putting in DKP for last night.  Thank you very much for your work on this Ptah!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×