Skip to content

feat(monsters): add compact MonstersTable backend and migrate plugins to API#4198

Draft
boscv wants to merge 3 commits intoOpenKore:masterfrom
boscv:montersTables
Draft

feat(monsters): add compact MonstersTable backend and migrate plugins to API#4198
boscv wants to merge 3 commits intoOpenKore:masterfrom
boscv:montersTables

Conversation

@boscv
Copy link
Copy Markdown
Contributor

@boscv boscv commented Mar 24, 2026

Context

This PR refactors how monsters_table is loaded and accessed by replacing direct %monstersTable usage with a dedicated MonstersTable API and introducing a compact backend with legacy fallback.

What changed

1) New module: src/MonstersTable.pm

  • Introduces a dedicated access layer with functions such as:
    • monster_exists, monster_field, monster_hp, monster_level
    • monster_is_looter_by_ai, monster_is_aggressive_by_ai
  • Adds compact in-memory structures:
    • %compact_rows (array-indexed records)
    • %compact_enums (deduplicated repeated strings like Size, Race, Element, Ai)
  • Adds AI mode caching (looter/aggressive) for fast checks.
  • Internal robustness updates:
    • numeric normalization via _as_number
    • removes magic AI index by using FIELD_INDEX{Ai}

2) Loading flow updates in src/functions.pl

  • monsters_table.txt is no longer parsed directly during addTableFile.
  • After Settings::loadAll, the system now:
    1. tries MonstersTable::load_compact_backend_from_file(...);
    2. falls back to legacy parse + initialize_compact_backend(...) if needed;
    3. logs source (direct-file or legacy-fallback).

3) Plugin migration to API (decoupling from global hash)

  • plugins/checkAggressive/checkAggressive.pl
    • removes duplicated local AI mode logic
    • uses monster_exists, monster_is_aggressive_by_ai, monster_level
  • plugins/checkLooter/checkLooter.pl
    • removes local AI constants and manual looter parser
    • uses monster_exists, monster_is_looter_by_ai
  • plugins/eCast/eCast.pl
    • migrates %monstersTable reads to monster_field/monster_hp/monster_level
    • fixes hasFreeCellBehind direction computation (targetPos->{y} usage)
    • renames callback to onPacketSkillUseNoDamage

Benefits

  • Lower coupling between plugins and internal monsters table representation.
  • Less duplicated AI logic across plugins.
  • Better memory optimization foundation with compact representation.
  • More resilient loading path with explicit fallback.

Notes

  • Current flow is still eager load at startup (compact backend + fallback), not true per-ID lazy loading.
  • Functional compatibility is preserved through fallback behavior.

Testing

  • perl -I src -c src/MonstersTable.pm
  • Ad-hoc Storable::nfreeze comparison (legacy vs compact representation) to estimate payload reduction.

Results (mean ± stddev)

Metric Legacy Compact Delta
Rows loaded 2555 2555 =
Build time (s) 0.028199 ± 0.004521 0.028755 ± 0.002800 0.98x (roughly equal)
Serialized size (bytes) 549,289 ± 0 120,734 ± 0 -78.02%
Lookup time (s) 0.126436 ± 0.014895 0.096869 ± 0.008839 1.31x faster
Run benchmark (bash + perl)
perl -Mstrict -Mwarnings -MTime::HiRes=time -MStorable=nfreeze -e '
my $runs = 10;
my $file="tables/monsters_table.txt";
open my $fh,"<",$file or die $!;
my @lines = <$fh>; close $fh;

sub build_legacy {
  my (%legacy,$rows);
  for my $line (@lines){ $line =~ s/^\s+|\s+$//g; next if $line eq q{} || $line =~ /^#/ || $line =~ /^ID\s+Level/i; my @f=split /\s+/,$line; next unless @f>=12; my ($id,$level,$hp,$ar,$sr,$ad,$am,$size,$race,$el,$ell,$cr,$ai)=@f;
    $legacy{$id}={Level=>$level,HP=>$hp,AttackRange=>$ar,SkillRange=>$sr,AttackDelay=>$ad,AttackMotion=>$am,Size=>$size,Race=>$race,Element=>$el,ElementLevel=>$ell,ChaseRange=>$cr,Ai=>$ai};
    $rows++;
  }
  return (\%legacy,$rows);
}

sub build_compact {
  my (%rows,%enums,%enum_ids,$n);
  my $eid = sub { my($t,$name)=@_; return undef unless defined $name; $enum_ids{$t} ||= {}; $enums{$t} ||= []; if(!exists $enum_ids{$t}{$name}){ push @{$enums{$t}}, $name; $enum_ids{$t}{$name} = $#{$enums{$t}}; } return $enum_ids{$t}{$name}; };
  for my $line (@lines){ $line =~ s/^\s+|\s+$//g; next if $line eq q{} || $line =~ /^#/ || $line =~ /^ID\s+Level/i; my @f=split /\s+/,$line; next unless @f>=12; my ($id,$level,$hp,$ar,$sr,$ad,$am,$size,$race,$el,$ell,$cr,$ai)=@f;
    $rows{$id}=[0+($level||0),0+($hp||0),0+($ar||0),0+($sr||0),0+($ad||0),0+($am||0),$eid->("Size",$size//"Small"),$eid->("Race",$race//"Formless"),$eid->("Element",$el//"Neutral"),0+($ell||1),0+($cr||0),$eid->("Ai",uc($ai//"06"))];
    $n++;
  }
  return ({rows=>\%rows,enums=>\%enums},$n);
}

sub mean { my $a=shift; my $s=0; $s+=$_ for @$a; return $s/@$a; }
sub stddev { my $a=shift; my $m=mean($a); my $s=0; $s+=($_-$m)**2 for @$a; return sqrt($s/@$a); }

my (@build_legacy,@build_compact,@freeze_legacy,@freeze_compact,@lookup_legacy,@lookup_compact);
my ($rows_legacy,$rows_compact);
for my $r (1..$runs){
  my $t0=time; my ($legacy,$n1)=build_legacy(); my $t1=time;
  my ($compact,$n2)=build_compact(); my $t2=time;
  $rows_legacy=$n1; $rows_compact=$n2;

  push @build_legacy, $t1-$t0;
  push @build_compact, $t2-$t1;
  push @freeze_legacy, length(nfreeze($legacy));
  push @freeze_compact, length(nfreeze($compact));

  my @ids = keys %{$legacy};
  my $iters = 200000;
  my $sum=0; my $s0=time; for(1..$iters){ my $id=$ids[int(rand(@ids))]; $sum += $legacy->{$id}{HP}; $sum += $legacy->{$id}{Level}; } my $s1=time;
  my $sum2=0; my $rows = $compact->{rows}; my $s2=time; for(1..$iters){ my $id=$ids[int(rand(@ids))]; my $row=$rows->{$id}; $sum2 += $row->[1]; $sum2 += $row->[0]; } my $s3=time;
  push @lookup_legacy, $s1-$s0;
  push @lookup_compact, $s3-$s2;
}

my $m_bl=mean(\@build_legacy); my $sd_bl=stddev(\@build_legacy);
my $m_bc=mean(\@build_compact); my $sd_bc=stddev(\@build_compact);
my $m_fl=mean(\@freeze_legacy); my $sd_fl=stddev(\@freeze_legacy);
my $m_fc=mean(\@freeze_compact); my $sd_fc=stddev(\@freeze_compact);
my $m_ll=mean(\@lookup_legacy); my $sd_ll=stddev(\@lookup_legacy);
my $m_lc=mean(\@lookup_compact); my $sd_lc=stddev(\@lookup_compact);

printf "runs=%d rows_legacy=%d rows_compact=%d\n",$runs,$rows_legacy,$rows_compact;
printf "build_legacy_mean_sec=%.6f sd=%.6f\n",$m_bl,$sd_bl;
printf "build_compact_mean_sec=%.6f sd=%.6f\n",$m_bc,$sd_bc;
printf "build_speedup_x=%.2f\n",($m_bl/$m_bc);
printf "freeze_legacy_mean_bytes=%.2f sd=%.2f\n",$m_fl,$sd_fl;
printf "freeze_compact_mean_bytes=%.2f sd=%.2f\n",$m_fc,$sd_fc;
printf "size_saving_pct=%.2f\n",100*(1-$m_fc/$m_fl);
printf "lookup_legacy_mean_sec=%.6f sd=%.6f\n",$m_ll,$sd_ll;
printf "lookup_compact_mean_sec=%.6f sd=%.6f\n",$m_lc,$sd_lc;
printf "lookup_speedup_x=%.2f\n",($m_ll/$m_lc);
'

boscv added 3 commits March 24, 2026 00:23
Introduce MonstersTable.pm to centralize access to monsters_table data and provide helper APIs (monster_exists, monster_field, monster_hp, monster_level, monster_is_looter_by_ai, monster_is_aggressive_by_ai, compact backend management, etc.).

Refactor plugins to use the new API instead of directly reading %monstersTable or duplicating AI flag logic:
- plugins/checkAggressive: use MonstersTable functions, remove local AI constant/flag parsing, and use monster_level() in debug messages.
- plugins/checkLooter: use monster_is_looter_by_ai(), log-warning path preserved, remove duplicated AI parsing.
- plugins/eCast: use monster_field()/monster_hp()/monster_level(), add improved packet message HP injection, track element changes, fix map_calc_dir_xy argument bug, and rename onPacketSkillUseNoDmg -> onPacketSkillUseNoDamage for clarity.

Startup change: initialize MonstersTable compact backend at load (purging legacy %monstersTable by default) via functions.pl to enable compact storage and precompute AI flags.

Overall this centralizes monster data handling, removes duplicated AI constants/logic, and enables a compact backend for performance and memory improvements.
Introduce MonstersTable::load_compact_backend_from_file to load compact-format monster data directly from a file (with optional purge_legacy). The new loader parses compact rows, builds internal compact_rows/enum structures, enables the compact backend, and rebuilds AI flags. monster_get now warns once and returns early when the compact backend is enabled, and reset_backend_state clears that warning flag. In loadDataFiles, switch the Settings table loader for monsters_table.txt to a no-op and attempt a direct compact load from the file; if that fails or yields no rows, fall back to legacy parsing and then initialize the compact backend. The startup message now reports the number of rows and the source (direct-file or legacy-fallback).
Introduce _as_number to coerce numeric fields with optional fallback and use it to initialize numeric monster fields in initialize_compact_backend and load_compact_backend_from_file. Also replace the hard-coded Ai field index (11) with $FIELD_INDEX{Ai} in _rebuild_ai_flags_cache. These changes normalize undefined/empty values, ensure consistent defaults (e.g. ElementLevel fallback to 1), and remove a magic array index.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant