#include <XrdPfcDirState.hh>

Inheritance diagram for XrdPfc::DirState:

Collaboration diagram for XrdPfc::DirState:

Public Types
typedef DsMap_t::iterator	DsMap_i

typedef std::map< std::string, DirState >	DsMap_t

Public Member Functions
	DirState ()
	Constructor.

	DirState (DirState *parent)

	DirState (DirState *parent, const std::string &dname)

int	count_dirs_to_level (int max_depth) const

DirState *	create_child (const std::string &dir)

void	dump_recursively (const char *name, int max_depth) const

DirState *	find_dir (const std::string &dir, bool create_subdirs)

DirState *	find_path (const std::string &path, int max_depth, bool parse_as_lfn, bool create_subdirs, DirState **last_existing_dir=nullptr)

DirState *	find_path_tok (PathTokenizer &pt, int pos, bool create_subdirs, DirState **last_existing_dir=nullptr)

int	generate_dir_path (std::string &result)

DirState *	get_parent ()

void	init ()

bool	is_empty_leaf () const

void	reset_sshot_stats ()

void	update_stats_and_usages (std::vector< DirState * > *reap_candidates=nullptr)

void	upward_propagate_initial_scan_usages ()

Public Member Functions inherited from XrdPfc::DirStateBase
	DirStateBase ()

	DirStateBase (const std::string &dname)

Public Attributes
int	m_depth

DirStats	m_here_stats

DirUsage	m_here_usage

DirState *	m_parent = nullptr

DirStats	m_recursive_subdir_stats

DirUsage	m_recursive_subdir_usage

long long	m_RecursiveStBlocksFuzz = 0
	sum over the subdirs, set by propagation

bool	m_scanned = false

DirStats	m_sshot_stats

long long	m_StBlocksFuzz = 0

DsMap_t	m_subdirs

Public Attributes inherited from XrdPfc::DirStateBase
std::string	m_dir_name

Detailed Description

Definition at line 56 of file XrdPfcDirState.hh.

Member Typedef Documentation

◆ DsMap_i

typedef DsMap_t::iterator XrdPfc::DirState::DsMap_i

Definition at line 59 of file XrdPfcDirState.hh.

◆ DsMap_t

typedef std::map<std::string, DirState> XrdPfc::DirState::DsMap_t

Definition at line 58 of file XrdPfcDirState.hh.

Constructor & Destructor Documentation

◆ DirState() [1/3]

DirState::DirState ( )

Constructor.

Definition at line 12 of file XrdPfcDirState.cc.

12 : m_parent(0), m_depth(0)

13{}

XrdPfc::DirState::m_parent

DirState * m_parent

Definition XrdPfcDirState.hh:77

XrdPfc::DirState::m_depth

int m_depth

Definition XrdPfcDirState.hh:79

References m_depth, and m_parent.

Referenced by DirState(), DirState(), create_child(), find_dir(), find_path(), find_path_tok(), get_parent(), and init().

Here is the caller graph for this function:

◆ DirState() [2/3]

DirState::DirState ( DirState * parent )

Constructor

Parameters

DirState parent directory

Definition at line 19 of file XrdPfcDirState.cc.

                                   :
   m_parent(parent),
   m_depth(m_parent->m_depth + 1)
{}

References DirState(), m_depth, m_parent, and parent().

Here is the call graph for this function:

◆ DirState() [3/3]

DirState::DirState	(	DirState *	parent,
		const std::string &	dname )

Constructor

Parameters

parent	parent DirState object
dname	name of this directory only, no slashes, no extras.

Definition at line 29 of file XrdPfcDirState.cc.

                                                           :
  DirStateBase(dname),
  m_parent(parent),
  m_depth(m_parent->m_depth + 1)
{}

References DirState(), XrdPfc::DirStateBase::DirStateBase(), m_depth, m_parent, and parent().

Here is the call graph for this function:

Member Function Documentation

◆ count_dirs_to_level()

int DirState::count_dirs_to_level ( int max_depth ) const

Definition at line 218 of file XrdPfcDirState.cc.

{
   int n_dirs = 1;
   if (m_depth < max_depth)
   {
      for (auto & [name, ds] : m_subdirs)
      {
         n_dirs += ds.count_dirs_to_level(max_depth);
      }
   }
   return n_dirs;
}

References m_depth, and m_subdirs.

Referenced by XrdPfc::ResourceMonitor::heart_beat(), and XrdPfc::ResourceMonitor::perform_purge_check().

Here is the caller graph for this function:

◆ create_child()

DirState * DirState::create_child ( const std::string & dir )

Internal function called from find_dir or find_path_tok

Parameters

dir	subdir name

Definition at line 39 of file XrdPfcDirState.cc.

{
   std::pair<DsMap_i, bool> ir = m_subdirs.insert(std::make_pair(dir, DirState(this, dir)));
   return &ir.first->second;
}

References DirState(), and m_subdirs.

Referenced by find_dir(), and find_path_tok().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ dump_recursively()

void DirState::dump_recursively	(	const char *	name,
		int	max_depth ) const

Recursive print of statistics. Called if defined in pfc configuration.

Definition at line 235 of file XrdPfcDirState.cc.

{
   printf("%*d %s usage_here=%lld usage_sub=%lld usage_total=%lld num_ios=%d duration=%d b_hit=%lld b_miss=%lld b_byps=%lld b_wrtn=%lld\n",
          2 + 2 * m_depth, m_depth, name,
          512 * m_here_usage.m_StBlocks, 512 * m_recursive_subdir_usage.m_StBlocks,
          512 * (m_here_usage.m_StBlocks + m_recursive_subdir_usage.m_StBlocks),
          // Since the last snapshot, this directory and below.
          m_sshot_stats.m_NumIos, m_sshot_stats.m_Duration,
          m_sshot_stats.m_BytesHit, m_sshot_stats.m_BytesMissed, m_sshot_stats.m_BytesBypassed,
          m_sshot_stats.m_BytesWritten);
 
   if (m_depth < max_depth)
   {
      for (auto & [name, ds] : m_subdirs)
      {
         ds.dump_recursively(name.c_str(), max_depth);
      }
   }
}

References m_depth, m_here_usage, m_recursive_subdir_usage, m_sshot_stats, and m_subdirs.

◆ find_dir()

DirState * DirState::find_dir	(	const std::string &	dir,
		bool	create_subdirs )

Non recursive function to find an entry in this directory only.

Parameters

dir	subdir name
bool	create the subdir in this DirsStat
create_subdirs	if true and the dir is not found, a new DirState child is created

Definition at line 98 of file XrdPfcDirState.cc.

{
   DsMap_i i = m_subdirs.find(dir);
 
   if (i != m_subdirs.end())
      return &i->second;
 
   if (create_subdirs)
      return create_child(dir);
 
   return nullptr;
}

References DirState(), create_child(), and m_subdirs.

Here is the call graph for this function:

◆ find_path()

DirState * DirState::find_path	(	const std::string &	path,
		int	max_depth,
		bool	parse_as_lfn,
		bool	create_subdirs,
		DirState **	last_existing_dir = nullptr )

Recursive function to find DirState with given absolute dir path

Parameters

path	full path to parse
max_depth	directory depth to which to descend (value < 0 means full descent)
parse_as_lfn
create_subdirs

Definition at line 82 of file XrdPfcDirState.cc.

{
   PathTokenizer pt(path, max_depth, parse_as_lfn);
 
   if (last_existing_dir)
      *last_existing_dir = this;
 
   return find_path_tok(pt, 0, create_subdirs, last_existing_dir);
}

References DirState(), and find_path_tok().

Referenced by XrdPfc::ResourceMonitor::process_queues().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ find_path_tok()

DirState * DirState::find_path_tok	(	PathTokenizer &	pt,
		int	pos,
		bool	create_subdirs,
		DirState **	last_existing_dir = nullptr )

Internal function called from find_path

Parameters

dir	subdir name

Definition at line 49 of file XrdPfcDirState.cc.

{
   if (pos == pt.get_n_dirs())
      return this;
 
   DirState *ds = nullptr;
 
   DsMap_i i = m_subdirs.find(pt.m_dirs[pos]);
 
   if (i != m_subdirs.end())
   {
      ds = &i->second;
      if (last_existing_dir)
         *last_existing_dir = ds;
   }
   else if (create_subdirs)
   {
      ds = create_child(pt.m_dirs[pos]);
   }
 
   if (ds)
      return ds->find_path_tok(pt, pos + 1, create_subdirs, last_existing_dir);
 
   return nullptr;
}

References DirState(), create_child(), find_path_tok(), XrdPfc::PathTokenizer::get_n_dirs(), XrdPfc::PathTokenizer::m_dirs, and m_subdirs.

Referenced by find_path(), and find_path_tok().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ generate_dir_path()

int DirState::generate_dir_path ( std::string & result )

Parent-recursively height-first append '/' and directory names to the result string. Returns total number of added characters.

Definition at line 116 of file XrdPfcDirState.cc.

{
   int n_chars = 0;
   if (m_parent) {
      n_chars += m_parent->generate_dir_path(result) + 1 + m_dir_name.length();
      result += '/';
      result += m_dir_name;
   }
   return n_chars;
}

References XrdPfc::DirStateBase::m_dir_name, and m_parent.

◆ get_parent()

DirState * XrdPfc::DirState::get_parent ( )

inline

Definition at line 97 of file XrdPfcDirState.hh.

97{ return m_parent; }

References DirState(), and m_parent.

Here is the call graph for this function:

◆ init()

void XrdPfc::DirState::init ( )

References DirState(), and parent().

Here is the call graph for this function:

◆ is_empty_leaf()

bool DirState::is_empty_leaf ( ) const

No subdirs, files or open files, as of the last propagation, and no opens since. m_NDirectories can undercount, so m_subdirs is checked as well.

Definition at line 201 of file XrdPfcDirState.cc.

{
   return m_subdirs.empty() &&
          m_here_usage.m_NDirectories == 0 && m_here_usage.m_NFiles == 0 &&
          m_here_usage.m_NFilesOpen == 0 &&
          m_here_stats.m_NFilesOpened == 0 && m_here_stats.m_NFilesCreated == 0;
}

References m_here_stats, m_here_usage, and m_subdirs.

◆ reset_sshot_stats()

void DirState::reset_sshot_stats ( )

Definition at line 209 of file XrdPfcDirState.cc.

{
   for (DsMap_i i = m_subdirs.begin(); i != m_subdirs.end(); ++i)
   {
      i->second.reset_sshot_stats();
   }
   m_sshot_stats.Reset();
}

References m_sshot_stats, and m_subdirs.

◆ update_stats_and_usages()

void DirState::update_stats_and_usages ( std::vector< DirState * > * reap_candidates = nullptr )

Upward propagate stats to parents, join last open/close timestamps, apply deltas / stats to usages, and reset the deltas. If reap_candidates is given, collect the empty leaves into it.

Definition at line 160 of file XrdPfcDirState.cc.

{
   m_RecursiveStBlocksFuzz = 0;
   for (auto & [name, sub_ds] : m_subdirs)
   {
      sub_ds.update_stats_and_usages(reap_candidates);
 
      m_RecursiveStBlocksFuzz += sub_ds.m_StBlocksFuzz + sub_ds.m_RecursiveStBlocksFuzz;
 
      m_recursive_subdir_stats.AddUp(sub_ds.m_recursive_subdir_stats);
      m_recursive_subdir_stats.AddUp(sub_ds.m_here_stats);
      // nothing to do for m_here_stats.
 
      // Join timestamps from daugters and their sub-daugter trees.
      m_recursive_subdir_usage.update_last_times(sub_ds.m_recursive_subdir_usage);
      m_recursive_subdir_usage.update_last_times(sub_ds.m_here_usage);
 
      // The parent clears a child's deltas once absorbed; the root's are cleared by
      // DataFsState::update_stats_and_usages().
      sub_ds.m_here_stats.Reset();
      sub_ds.m_recursive_subdir_stats.Reset();
 
      if (reap_candidates && sub_ds.is_empty_leaf())
         reap_candidates->push_back(&sub_ds);
   }
 
   // Apply current deltas to sshot_stats (and, if eventually needed, purge_stats).
   m_sshot_stats.AddUp(m_recursive_subdir_stats);
   m_sshot_stats.AddUp(m_here_stats);
 
   // Apply deltas / stats to usages.
   m_here_usage.update_from_stats(m_here_stats);
   m_recursive_subdir_usage.update_from_stats(m_recursive_subdir_stats);
 
   // Once we have m_report_stats and m_purge_stats, update them here.
}

References m_here_stats, m_here_usage, m_recursive_subdir_stats, m_recursive_subdir_usage, m_RecursiveStBlocksFuzz, m_sshot_stats, and m_subdirs.

◆ upward_propagate_initial_scan_usages()

void DirState::upward_propagate_initial_scan_usages ( )

Propagate usages to parents after initial directory scan. Called from ResourceMonitor::perform_initial_scan()

Definition at line 131 of file XrdPfcDirState.cc.

{
   DirUsage &here    = m_here_usage;
   DirUsage &subdirs = m_recursive_subdir_usage;
 
   for (auto & [name, daughter] : m_subdirs)
   {
      daughter.upward_propagate_initial_scan_usages();
 
      DirUsage &dhere    = daughter.m_here_usage;
      DirUsage &dsubdirs = daughter.m_recursive_subdir_usage;
 
      here.m_NDirectories += 1;
 
      subdirs.m_StBlocks     += dhere.m_StBlocks     + dsubdirs.m_StBlocks;
      subdirs.m_NFiles       += dhere.m_NFiles       + dsubdirs.m_NFiles;
      subdirs.m_NDirectories += dhere.m_NDirectories + dsubdirs.m_NDirectories;
 
      // Carry the last-access times up, as update_stats_and_usages() does.
      subdirs.update_last_times(dhere);
      subdirs.update_last_times(dsubdirs);
   }
}

References m_here_usage, XrdPfc::DirUsage::m_NDirectories, XrdPfc::DirUsage::m_NFiles, m_recursive_subdir_usage, XrdPfc::DirUsage::m_StBlocks, m_subdirs, and XrdPfc::DirUsage::update_last_times().

Referenced by XrdPfc::ResourceMonitor::perform_initial_scan().

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

◆ m_depth

int XrdPfc::DirState::m_depth

Definition at line 79 of file XrdPfcDirState.hh.

Referenced by DirState(), DirState(), DirState(), count_dirs_to_level(), dump_recursively(), XrdPfc::ResourceMonitor::fill_pshot_vec_children(), and XrdPfc::ResourceMonitor::fill_sshot_vec_children().

◆ m_here_stats

DirStats XrdPfc::DirState::m_here_stats

Definition at line 61 of file XrdPfcDirState.hh.

Referenced by is_empty_leaf(), XrdPfc::note_file_open(), XrdPfc::ResourceMonitor::process_queues(), and update_stats_and_usages().

◆ m_here_usage

DirUsage XrdPfc::DirState::m_here_usage

Definition at line 64 of file XrdPfcDirState.hh.

Referenced by dump_recursively(), XrdPfc::ResourceMonitor::heart_beat(), is_empty_leaf(), XrdPfc::ResourceMonitor::perform_initial_scan(), XrdPfc::ResourceMonitor::perform_purge_check(), XrdPfc::ResourceMonitor::process_queues(), XrdPfc::ResourceMonitor::scan_dir_and_recurse(), update_stats_and_usages(), and upward_propagate_initial_scan_usages().

◆ m_parent

DirState* XrdPfc::DirState::m_parent = nullptr

Definition at line 77 of file XrdPfcDirState.hh.

Referenced by DirState(), DirState(), DirState(), generate_dir_path(), and get_parent().

◆ m_recursive_subdir_stats

DirStats XrdPfc::DirState::m_recursive_subdir_stats

Definition at line 62 of file XrdPfcDirState.hh.

Referenced by update_stats_and_usages().

◆ m_recursive_subdir_usage

DirUsage XrdPfc::DirState::m_recursive_subdir_usage

Definition at line 65 of file XrdPfcDirState.hh.

Referenced by dump_recursively(), XrdPfc::ResourceMonitor::heart_beat(), XrdPfc::ResourceMonitor::perform_initial_scan(), XrdPfc::ResourceMonitor::perform_purge_check(), update_stats_and_usages(), and upward_propagate_initial_scan_usages().

◆ m_RecursiveStBlocksFuzz

long long XrdPfc::DirState::m_RecursiveStBlocksFuzz = 0

sum over the subdirs, set by propagation

Definition at line 75 of file XrdPfcDirState.hh.

Referenced by update_stats_and_usages().

◆ m_scanned

bool XrdPfc::DirState::m_scanned = false

Definition at line 80 of file XrdPfcDirState.hh.

Referenced by XrdPfc::ResourceMonitor::scan_dir_and_recurse().

◆ m_sshot_stats

DirStats XrdPfc::DirState::m_sshot_stats

Definition at line 69 of file XrdPfcDirState.hh.

Referenced by dump_recursively(), reset_sshot_stats(), and update_stats_and_usages().

◆ m_StBlocksFuzz

long long XrdPfc::DirState::m_StBlocksFuzz = 0

Accumulated magnitude of st_blocks corrections made at this node, never reset; a reaped child's is carried here. A diagnostic of accounting drift.

Definition at line 74 of file XrdPfcDirState.hh.

◆ m_subdirs

DsMap_t XrdPfc::DirState::m_subdirs

Definition at line 78 of file XrdPfcDirState.hh.

Referenced by count_dirs_to_level(), create_child(), dump_recursively(), XrdPfc::ResourceMonitor::fill_pshot_vec_children(), XrdPfc::ResourceMonitor::fill_sshot_vec_children(), find_dir(), find_path_tok(), is_empty_leaf(), reset_sshot_stats(), update_stats_and_usages(), and upward_propagate_initial_scan_usages().

The documentation for this struct was generated from the following files:

Public Types

Public Member Functions

Public Attributes

Detailed Description

Member Typedef Documentation

◆ DsMap_i

◆ DsMap_t

Constructor & Destructor Documentation

◆ DirState() [1/3]

◆ DirState() [2/3]

◆ DirState() [3/3]

Member Function Documentation

◆ count_dirs_to_level()

◆ create_child()

◆ dump_recursively()

◆ find_dir()

◆ find_path()

◆ find_path_tok()

◆ generate_dir_path()

◆ get_parent()

◆ init()

◆ is_empty_leaf()

◆ reset_sshot_stats()

◆ update_stats_and_usages()

◆ upward_propagate_initial_scan_usages()

Member Data Documentation

◆ m_depth

◆ m_here_stats

◆ m_here_usage

◆ m_parent

◆ m_recursive_subdir_stats

◆ m_recursive_subdir_usage

◆ m_RecursiveStBlocksFuzz

◆ m_scanned

◆ m_sshot_stats

◆ m_StBlocksFuzz

◆ m_subdirs