ComputationCache Class Reference

Class ComputationCache is used inside class CachingOptimizingCompiler to cache previously computed computations. More...

#include <nnet-optimize-utils.h>

Collaboration diagram for ComputationCache:

Public Member Functions
	ComputationCache (int32 cache_capacity)
void	Read (std::istream &is, bool binary)
void	Write (std::ostream &os, bool binary) const
std::shared_ptr< const NnetComputation >	Find (const ComputationRequest &request)
std::shared_ptr< const NnetComputation >	Insert (const ComputationRequest &request, const NnetComputation *computation)
	~ComputationCache ()
void	Check (const Nnet &nnet) const

Private Types
typedef std::list< const ComputationRequest * >	AqType
typedef unordered_map< const ComputationRequest *, std::pair< std::shared_ptr< const NnetComputation >, AqType::iterator >, ComputationRequestHasher, ComputationRequestPtrEqual >	CacheType

Private Attributes
std::mutex	mutex_
int32	cache_capacity_
AqType	access_queue_
CacheType	computation_cache_

Detailed Description

Class ComputationCache is used inside class CachingOptimizingCompiler to cache previously computed computations.

The code was moved from class CachingOptimizingCompiler to this separate class for clarity when adding thread-safety functionality. It's OK to call Find() and Insert() from multiple threads without additional synchronization.

Definition at line 625 of file nnet-optimize-utils.h.

Member Typedef Documentation

AqType

typedef std::list<const ComputationRequest*> AqType

private

Definition at line 668 of file nnet-optimize-utils.h.

CacheType

typedef unordered_map<const ComputationRequest*, std::pair<std::shared_ptr<const NnetComputation>, AqType::iterator>, ComputationRequestHasher, ComputationRequestPtrEqual> CacheType

private

Definition at line 677 of file nnet-optimize-utils.h.

Constructor & Destructor Documentation

ComputationCache()

ComputationCache

(

int32

cache_capacity

)

Definition at line 4969 of file nnet-optimize-utils.cc.

References KALDI_ASSERT.

                                                      :
    cache_capacity_(cache_capacity) {
  KALDI_ASSERT(cache_capacity > 0);
}

~ComputationCache()

~ComputationCache

(

)

Definition at line 5062 of file nnet-optimize-utils.cc.

References ComputationCache::computation_cache_.

                                    {
  CacheType::const_iterator iter = computation_cache_.begin(),
      end = computation_cache_.end();
  // We only need to explicitly delete the pointer to the ComputationRequest.
  // The pointers to Computation are deleted automatically by std::shared_ptr
  // when the reference count goes to zero.
  for (; iter != end; ++iter)
    delete iter->first;
}

Member Function Documentation

Check()

void Check

(

const Nnet &

nnet

)

const

Definition at line 5037 of file nnet-optimize-utils.cc.

References ComputationChecker::Check(), and ComputationCache::computation_cache_.

Referenced by CachingOptimizingCompiler::ReadCache().

                                                   {
  CacheType::const_iterator iter = computation_cache_.begin(),
      end = computation_cache_.end();
  // We only need to explicitly delete the pointer to the ComputationRequest.
  // The pointers to Computation are deleted automatically by std::shared_ptr
  // when the reference count goes to zero.
  for (; iter != end; ++iter) {
    const NnetComputation &computation = *(iter->second.first);
    CheckComputationOptions check_config;
    ComputationChecker checker(check_config, nnet, computation);
    checker.Check();
  }
}

Find()

std::shared_ptr< const NnetComputation > Find

(

const ComputationRequest &

request

)

Definition at line 4951 of file nnet-optimize-utils.cc.

Referenced by CachingOptimizingCompiler::CompileInternal().

                                          {
  std::lock_guard<std::mutex> lock(mutex_);

  CacheType::iterator iter = computation_cache_.find(&in_request);
  if (iter == computation_cache_.end()) {
    return NULL;
  } else {
    std::shared_ptr<const NnetComputation> ans = iter->second.first;
    // Update access record by moving the accessed request to the end of the
    // access queue, which declares that it's the most recently used.
    access_queue_.splice(access_queue_.end(), access_queue_,
                         iter->second.second);
    return ans;
  }
}

Insert()

std::shared_ptr< const NnetComputation > Insert	(	const ComputationRequest &	request,
		const NnetComputation *	computation
	)

Definition at line 4974 of file nnet-optimize-utils.cc.

References ComputationCache::access_queue_, ComputationCache::cache_capacity_, ComputationCache::computation_cache_, KALDI_ASSERT, and ComputationCache::mutex_.

Referenced by CachingOptimizingCompiler::CompileInternal(), and ComputationCache::Read().

                                           {

  std::lock_guard<std::mutex> lock(mutex_);
  if (static_cast<int32>(computation_cache_.size()) >= cache_capacity_) {
    //  Cache has reached capacity; purge the least-recently-accessed request
    const CacheType::iterator iter =
        computation_cache_.find(access_queue_.front());
    KALDI_ASSERT(iter != computation_cache_.end());
    const ComputationRequest *request = iter->first;
    computation_cache_.erase(iter);
    delete request;
    // we don't need to delete the computation in iter->second.first, as the
    // shared_ptr takes care of that automatically.
    access_queue_.pop_front();
  }

  // Now insert the thing we need to insert.  We'll own the pointer 'request' in
  // 'computation_cache_', so we need to allocate our own version.
  ComputationRequest *request = new ComputationRequest(request_in);
  // When we construct this shared_ptr, it takes ownership of the pointer
  // 'computation_in'.
  std::shared_ptr<const NnetComputation> computation(computation_in);

  AqType::iterator ait = access_queue_.insert(access_queue_.end(), request);

  std::pair<CacheType::iterator, bool> p = computation_cache_.insert(
      std::make_pair(request, std::make_pair(computation, ait)));
  if (!p.second) {
    // if p.second is false, this pair was not inserted because
    // a computation for the same computation-request already existed in
    // the map. This is possible in multi-threaded operations, if two
    // threads try to compile the same computation at the same time (only
    // one of them will successfully add it).
    // We need to erase the access-queue element that we just added, it's
    // no longer going to be needed.
    access_queue_.erase(ait);
    delete request;
  }
  return computation;
}

Read()

void Read	(	std::istream &	is,
		bool	binary
	)

Definition at line 5018 of file nnet-optimize-utils.cc.

References ComputationCache::access_queue_, ComputationCache::computation_cache_, kaldi::nnet3::ExpectToken(), ComputationCache::Insert(), KALDI_ASSERT, kaldi::nnet2::NnetComputation(), ComputationRequest::Read(), NnetComputation::Read(), and kaldi::ReadBasicType().

Referenced by CachingOptimizingCompiler::ReadCache().

                                                       {
  // Note: the object on disk doesn't have tokens like "<ComputationCache>"
  // and "</ComputationCache>" for back-compatibility reasons.
  int32 computation_cache_size;
  ExpectToken(is, binary, "<ComputationCacheSize>");
  ReadBasicType(is, binary, &computation_cache_size);
  KALDI_ASSERT(computation_cache_size >= 0);
  computation_cache_.clear();
  access_queue_.clear();
  ExpectToken(is, binary, "<ComputationCache>");
  for (size_t c = 0; c < computation_cache_size; c++) {
    ComputationRequest request;
    request.Read(is, binary);
    NnetComputation *computation = new NnetComputation();
    computation->Read(is, binary);
    Insert(request, computation);
  }
}

Write()

void Write	(	std::ostream &	os,
		bool	binary
	)		const

Definition at line 5051 of file nnet-optimize-utils.cc.

References ComputationCache::computation_cache_, kaldi::WriteBasicType(), and kaldi::WriteToken().

Referenced by CachingOptimizingCompiler::WriteCache().

                                                              {
  WriteToken(os, binary, "<ComputationCacheSize>");
  WriteBasicType(os, binary, static_cast<int32>(computation_cache_.size()));
  WriteToken(os, binary, "<ComputationCache>");
  for (CacheType::const_iterator iter = computation_cache_.begin();
           iter != computation_cache_.end(); ++iter) {
    iter->first->Write(os, binary);
    iter->second.first->Write(os, binary);
  }
}

Member Data Documentation

access_queue_

AqType access_queue_

private

Definition at line 669 of file nnet-optimize-utils.h.

Referenced by ComputationCache::Insert(), and ComputationCache::Read().

cache_capacity_

int32 cache_capacity_

private

Definition at line 660 of file nnet-optimize-utils.h.

Referenced by ComputationCache::Insert().

computation_cache_

CacheType computation_cache_

private

Definition at line 678 of file nnet-optimize-utils.h.

Referenced by ComputationCache::Check(), ComputationCache::Insert(), ComputationCache::Read(), ComputationCache::Write(), and ComputationCache::~ComputationCache().

mutex_

std::mutex mutex_

private

Definition at line 658 of file nnet-optimize-utils.h.

Referenced by ComputationCache::Insert().

---

The documentation for this class was generated from the following files:

• nnet3/nnet-optimize-utils.h
• nnet3/nnet-optimize-utils.cc