Skip to content

Commit

Permalink
Migrate export to use streaming
Browse files Browse the repository at this point in the history
  • Loading branch information
whitfin committed Sep 14, 2024
1 parent 4e5164f commit 7b9cd09
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 3 deletions.
14 changes: 12 additions & 2 deletions lib/cachex/actions/export.ex
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ defmodule Cachex.Actions.Export do
# This command is extremely expensive as it turns the entire cache table into
# a list, and so should be used sparingly. It's provided purely because it's
# the backing implementation of the `dump/3` command.
alias Cachex.Actions.Stream, as: CachexStream
alias Cachex.Query

# add required imports
import Cachex.Spec

##############
Expand All @@ -20,6 +24,12 @@ defmodule Cachex.Actions.Export do
This action should only be used in the case of exports and/or debugging, due
to the memory overhead involved, as well as the large concatenations.
"""
def execute(cache(name: name), _options),
do: {:ok, :ets.tab2list(name)}
def execute(cache() = cache, options) do
query = Query.create()
batch = Keyword.take(options, [:batch_size])

with {:ok, stream} <- CachexStream.execute(cache, query, batch) do
{:ok, Enum.to_list(stream)}
end
end
end
8 changes: 7 additions & 1 deletion lib/cachex/actions/stream.ex
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,13 @@ defmodule Cachex.Actions.Stream do

# we're starting!
:"$start_of_table" ->
:ets.select(name, spec, batch)
case :ets.select(name, spec, batch) do
:"$end_of_table" ->
{:halt, nil}

continuation ->
continuation
end

# we're continuing!
continuation ->
Expand Down

0 comments on commit 7b9cd09

Please sign in to comment.