How to stop evergrowing disk usage by Stellar node
How to stop evergrowing disk usage by Stellar node
Loading saved threads...
hotbit · External communityPost link
External question — Stellar Stack Exchange
Author: hotbit
Original post: https://stellar.stackexchange.com/questions/5773
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
Disk usage is growing steadily and we cannot find the solution. There are two questions from about 2-3 years ago:
Stop Horizon/Stellar node growing disk usage
Pruning and keeping disk usage low
But in neither of them is the solution clear.
Our core setting:
CATCHUP_RECENT=100
AUTOMATIC_MAINTENANCE_PERIOD=60
CATCHUP_COMPLETE=false
Postgresql settings:
# AUTOVACUUM PARAMETERS
#------------------------------------------------------------------------------
autovacuum = on # Enable autovacuum subprocess? 'on'
# requires track_counts to also be on.
#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and
# their durations, > 0 logs only
# actions running at least this number
# of milliseconds.
autovacuum_max_workers = 2 # max number of autovacuum subprocesses
# (change requires restart)
autovacuum_naptime = 1 # time between autovacuum runs
autovacuum_vacuum_threshold = 50 # min number of row updates before
# vacuum
#autovacuum_analyze_threshold = 50 # min number of row updates before
# analyze
autovacuum_vacuum_scale_factor = 0.01 # fraction of table size before vacuum
#autovacuum_analyze_scale_factor = 0.01 # fraction of table size before analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum
# (change requires restart)
#autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age
# before forced vacuum
# (change requires restart)
autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for
# autovacuum, in milliseconds;
# -1 means use vacuum_cost_delay
autovacuum_vacuum_cost_limit = 1000 # default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit
Ledger history count in horizon.env is set to and not growing (actual count stabilized and when checked is ~18500):
HISTORY_RETENTION_COUNT="17280"
Autovacuum workers seem to be working, but the disk usage is still growing by several GB daily.
Quote
Report
Graydon Hoare · External communityPost link
External answer — Stellar Stack Exchange
Author: Graydon Hoare
Original post: https://stellar.stackexchange.com/a/5775
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
What is
AUTOMATIC_MAINTENANCE_COUNT
set to in core?
Can you check on the sizes of individual tables? Like if you connect to postgres using
psql
, what do you see for
select count(*) from txhistory
or
select count(*) from publishqueue
?
When you say "disk usage" is increasing, are you sure it is postgres disk usage and not (say) archive disk usage or bucket disk usage? Which directories are growing and what are their current sizes?
Quote
Report
Post Reply
Quoted from Forex.com.bd-Editorial External question — Stellar Stack Exchange Author: hotbit Source score (net votes, not local likes): 1 Original post: https://stellar.stackexchange.com/questions/5773 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Disk usage is growing steadily and we cannot find the solution. There are two questions from about 2-3 years ago: Stop Horizon/Stellar node growing disk usage Pruning and keeping disk usage low But in neither of them is the solution clear. Our core setting: CATCHUP_RECENT=100 AUTOMATIC_MAINTENANCE_PERIOD=60 CATCHUP_COMPLETE=false Postgresql settings: # AUTOVACUUM PARAMETERS #------------------------------------------------------------------------------ autovacuum = on # Enable autovacuum subprocess? 'on' # requires track_counts to also be on. #log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and # their durations, > 0 logs only # actions running at least this number # of milliseconds. autovacuum_max_workers = 2 # max number of autovacuum subprocesses # (change requires restart) autovacuum_naptime = 1 # time between autovacuum runs autovacuum_vacuum_threshold = 50 # min number of row updates before # vacuum #autovacuum_analyze_threshold = 50 # min number of row updates before # analyze autovacuum_vacuum_scale_factor = 0.01 # fraction of table size before vacuum #autovacuum_analyze_scale_factor = 0.01 # fraction of table size before analyze #autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum # (change requires restart) #autovacuum_multixact_freeze_max_age = 400000000 # maximum multixact age # before forced vacuum # (change requires restart) autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for # autovacuum, in milliseconds; # -1 means use vacuum_cost_delay autovacuum_vacuum_cost_limit = 1000 # default vacuum cost limit for # autovacuum, -1 means use # vacuum_cost_limit Ledger history count in horizon.env is set to and not growing (actual count stabilized and when checked is ~18500): HISTORY_RETENTION_COUNT="17280" Autovacuum workers seem to be working, but the disk usage is still growing by several GB daily.
Checking account access…