Bug: sam local invoke "Cannot find module 'index'" (happens when running from docker container only)

[osotnikov]

Description:

When doing a sam build followed by a sam local invoke on a simple hello world lambda (nodejs16) everything works fine on windows but fails on linux (docker container, ubuntu:jammy-20220801).

Steps to reproduce:

Install nodejs 16 (although same observed on 20 as well). Create a small helloWorld.js lambda, create a basic template.yaml, run sam build, run sam local invoke.

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Description: A simple Hello World Serverless project
Resources:
  HelloWorld:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs16.x
      Handler: index.handler
      CodeUri: .

`'use strict';

// A simple hello world Lambda function
exports.handler = (event, context, callback) => {

    console.log('LOG: Name is ' + event.name);
    callback(null, "Hello " + event.name);

}

Observed result:

On windows:

On jammy docker container:

same happens with node 16.13 as well...

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

docker container works on the same folder as windows with parent folder being mounted as volume.

version: "3.9"
services:
  stockmonitor:
    container_name: stockmonitor-infrastructure
    image: stockmonitor_main
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    volumes:
      - ../..:/app
      - /var/run/docker.sock:/var/run/docker.sock
    network_mode: host

[hnnasit]

Hi @osotnikov, I tried reproducing this on a linux machine but got the same output as the Windows one you have.

/var/task:ro,delegated, inside runtime container                                
START RequestId: 58b22b2b-d41b-41e9-85b2-4905c4a6aa85 Version: $LATEST
2024-01-21T19:09:59.851Z	35bf7d3f-e218-4067-8569-2e876fb3e5ea	INFO	LOG: Name is undefined
END RequestId: 35bf7d3f-e218-4067-8569-2e876fb3e5ea
REPORT RequestId: 35bf7d3f-e218-4067-8569-2e876fb3e5ea	Init Duration: 0.03 ms	Duration: 77.78 ms	Billed Duration: 78 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	
"Hello undefined"

This might be an issue with the container configuration. Will try installing SAM CLI and reproducing the behavior using ubuntu:jammy image.

[osotnikov]

I found what the issue is. Basically I am running sam local invoke from within a docker container. To support sam local I've installed a docker client within my docker container and mounted /var/run/docker.sock:/var/run/docker.sock. I basically have docker outside of docker setup now. sam local invoke relies on spinning up a docker container itself, so it spins up a sibling container to execute. This container of course does not have access to my container's filesystem. The best I can do is to call sam like this: sam local invoke --docker-volume-basedir '/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dist' (host is windows). I guess I'll just pass my hosts path as an ENV variable into the dockerfile and change create a script to call sam local.

This is a similar issue to mine: #4589

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[osotnikov]

Sorry, tried it again today and having the same issue as before, it's working on windows but not from sam local invoked in a docker container:

root@docker-desktop:/app/lambdas/feature/helloWorldLambda/dist# sam local invoke --docker-volume-b
asedir '/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda
/dist'
Invoking index.handler (nodejs20.x)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:20-rapid-x86_64.

Mounting
/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dist/He
lloWorldFunction as /var/task:ro,delegated, inside runtime container

Error: [Errno 2] No such file or directory: '/c/dev/repos/stockmonitor/stockmonitor-aws-infrastruc
ture/lambdas/feature/helloWorldLambda/dist/HelloWorldFunction'
Traceback:
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "samcli/cli/cli_config_file.py", line 347, in wrapper
  File "click/decorators.py", line 92, in new_func
  File "click/core.py", line 783, in invoke
  File "samcli/lib/telemetry/metric.py", line 184, in wrapped
  File "samcli/lib/telemetry/metric.py", line 149, in wrapped
  File "samcli/lib/utils/version_checker.py", line 42, in wrapped
  File "samcli/cli/main.py", line 95, in wrapper
  File "samcli/commands/local/invoke/cli.py", line 103, in cli
  File "samcli/commands/local/invoke/cli.py", line 201, in do_cli
  File "samcli/commands/local/lib/local_lambda.py", line 144, in invoke
  File "samcli/lib/telemetry/metric.py", line 324, in wrapped_func
  File "samcli/local/lambdafn/runtime.py", line 192, in invoke
  File "samcli/local/lambdafn/runtime.py", line 107, in create
  File "samcli/local/docker/manager.py", line 96, in create
  File "samcli/local/docker/container.py", line 167, in create
  File "samcli/local/docker/container.py", line 250, in _create_mapped_symlink_files

An unexpected error was encountered while executing "sam local invoke".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20local%20invoke%20-
%20FileNotFoundError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20local%20
invoke%20-%20FileNotFoundError
root@docker-desktop:/app/lambdas/feature/helloWorldLambda/dist#

For some reason it's appending HelloWorldFunction at the end of the path it mounts. HelloWorldFunction is the name of my function in the template but dist folder has no folder of such name. The same happens even if I create HelloWorldFunction, change CodUri accordingly and move index.js to that folder. Everything works fine on the windows host though. Maybe my python was updated since I just download the latest in my Dockerfile. I don't think sam was updated though.

Same thing happens when I call

sam local invoke --container-host $(docker network inspect bridge | jq -r '.[0].IPAM.Config[0].Gateway') --container-host-interface 0.0.0.0 --docker-volume-basedir $PROJECT_BASE_IN_HOST/lambdas/feature/helloWorldLambda/dist --debug

or

sam local invoke --container-host docker --container-host-interface 0.0.0.0 --docker-volume-basedir $PROJECT_BASE_IN_HOST/lambdas/feature/helloWorldLambda/dist --debug

with output

root@docker-desktop:/app/lambdas/feature/helloWorldLambda/dist# yarn run sl-invoke2
yarn run v1.22.19
warning ../../../package.json: No license field
$ sam local invoke --container-host docker --container-host-interface 0.0.0.0 --docker-volume-base
dir $PROJECT_BASE_IN_HOST/lambdas/feature/helloWorldLambda/dist --debug
2024-01-30 11:12:52,677 | No config file found in this directory.
2024-01-30 11:12:52,681 | OSError occurred while reading TOML file: [Errno 2] No such file or
directory: '/app/lambdas/feature/helloWorldLambda/samconfig.toml'
2024-01-30 11:12:52,683 | Config file location:
/app/lambdas/feature/helloWorldLambda/samconfig.toml
2024-01-30 11:12:52,684 | Config file '/app/lambdas/feature/helloWorldLambda/samconfig.toml' does
not exist
2024-01-30 11:12:52,692 | Using SAM Template at /app/lambdas/feature/helloWorldLambda/template.yml
2024-01-30 11:12:52,712 | OSError occurred while reading TOML file: [Errno 2] No such file or
directory: '/app/lambdas/feature/helloWorldLambda/samconfig.toml'
2024-01-30 11:12:52,713 | Using config file: samconfig.toml, config environment: default
2024-01-30 11:12:52,714 | Expand command line arguments to:
2024-01-30 11:12:52,715 | --template_file=/app/lambdas/feature/helloWorldLambda/template.yml
--container_host=docker --container_host_interface=0.0.0.0
--docker_volume_basedir=/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/
helloWorldLambda/dist --no_event --layer_cache_basedir=/root/.aws-sam/layers-pkg
2024-01-30 11:12:52,717 | local invoke command is called
2024-01-30 11:12:52,723 | Collected default values for parameters: {'env': 'dev'}
2024-01-30 11:12:52,774 | There is no customer defined id or cdk path defined for resource
StockMonitorApi, so we will use the resource logical id as the resource id
2024-01-30 11:12:52,775 | There is no customer defined id or cdk path defined for resource
HelloWorldFunction, so we will use the resource logical id as the resource id
2024-01-30 11:12:52,776 | Unable to resolve property DependsOn: None. Leaving as is.
2024-01-30 11:12:52,777 | 0 stacks found in the template
2024-01-30 11:12:52,778 | Collected default values for parameters: {'env': 'dev'}
2024-01-30 11:12:52,796 | There is no customer defined id or cdk path defined for resource
StockMonitorApi, so we will use the resource logical id as the resource id
2024-01-30 11:12:52,797 | There is no customer defined id or cdk path defined for resource
HelloWorldFunction, so we will use the resource logical id as the resource id
2024-01-30 11:12:52,799 | Unable to resolve property DependsOn: None. Leaving as is.
2024-01-30 11:12:52,799 | 2 resources found in the stack
2024-01-30 11:12:52,800 | Found Serverless function with name='HelloWorldFunction' and CodeUri='.'
2024-01-30 11:12:52,808 | Found one Lambda function with name 'HelloWorldFunction'
2024-01-30 11:12:52,810 | Invoking index.handler (nodejs20.x)
2024-01-30 11:12:52,811 | No environment variables found for function 'HelloWorldFunction'
2024-01-30 11:12:52,812 | Loading AWS credentials from session with profile 'None'
2024-01-30 11:12:52,823 | Resolving code path.
Cwd=/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dis
t, CodeUri=.
2024-01-30 11:12:52,824 | Resolved absolute path to code is
/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dist
2024-01-30 11:12:52,825 | Code
/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dist is
not a zip/jar file
2024-01-30 11:12:55,106 | Local image is up-to-date
2024-01-30 11:12:55,113 | Checking free port on 0.0.0.0:7434
2024-01-30 11:12:55,117 | Using local image: public.ecr.aws/lambda/nodejs:20-rapid-x86_64.

2024-01-30 11:12:55,118 | Mounting
/c/dev/repos/stockmonitor/stockmonitor-aws-infrastructure/lambdas/feature/helloWorldLambda/dist as
/var/task:ro,delegated, inside runtime container
2024-01-30 11:12:55,119 | Cleaning all decompressed code dirs
2024-01-30 11:12:55,120 | Telemetry endpoint configured to be
https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-01-30 11:12:55,133 | Telemetry endpoint configured to be
https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-01-30 11:12:55,134 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId':
'55c6ab8e-3fd4-4976-ac7b-44d74459a216', 'installationId': '4e1a5372-7698-4b99-925d-88bd0b7a0176',
'sessionId': '8d562ff3-b187-41bd-ad82-7c3a66d6465d', 'executionEnvironment': 'CLI', 'ci': False,
'pyversion': '3.11.3', 'samcliVersion': '1.107.0', 'awsProfileProvided': False,
'debugFlagProvided': True, 'region': '', 'commandName': 'sam local invoke',
'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName':
'5d8fe68e9c78dc766c985409684bed5bb7f8770afd84b85a225a58c6b30b2e3b', 'initialCommit': None},
'duration': 2406, 'exitReason': 'FileNotFoundError', 'exitCode': 255}}]}
2024-01-30 11:12:55,134 | Unable to find Click Context for getting session_id.
2024-01-30 11:12:55,139 | Sending Telemetry: {'metrics': [{'events': {'requestId':
'ec7575f5-74d8-4f77-9838-28260cea8508', 'installationId': '4e1a5372-7698-4b99-925d-88bd0b7a0176',
'sessionId': '8d562ff3-b187-41bd-ad82-7c3a66d6465d', 'executionEnvironment': 'CLI', 'ci': False,
'pyversion': '3.11.3', 'samcliVersion': '1.107.0', 'commandName': 'sam local invoke',
'metricSpecificAttributes': {'events': [{'event_name': 'SamConfigFileExtension', 'event_value':
'.toml', 'thread_id': '8609f57a098a428b8e2055cd30266caa', 'time_stamp': '2024-01-30 11:12:52.683',
'exception_name': None}, {'event_name': 'SamConfigFileExtension', 'event_value': '.toml',
'thread_id': '66be564d17a54ed78be41b78cef46aa5', 'time_stamp': '2024-01-30 11:12:52.713',
'exception_name': None}]}}}]}
2024-01-30 11:12:55,648 |
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read
timed out. (read timeout=0.1)
2024-01-30 11:12:55,650 |
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read
timed out. (read timeout=0.1)

Error: [Errno 2] No such file or directory: '/c/dev/repos/stockmonitor/stockmonitor-aws-infrastruc
ture/lambdas/feature/helloWorldLambda/dist'
Traceback:
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "samcli/cli/cli_config_file.py", line 347, in wrapper
  File "click/decorators.py", line 92, in new_func
  File "click/core.py", line 783, in invoke
  File "samcli/lib/telemetry/metric.py", line 184, in wrapped
  File "samcli/lib/telemetry/metric.py", line 149, in wrapped
  File "samcli/lib/utils/version_checker.py", line 42, in wrapped
  File "samcli/cli/main.py", line 95, in wrapper
  File "samcli/commands/local/invoke/cli.py", line 103, in cli
  File "samcli/commands/local/invoke/cli.py", line 201, in do_cli
  File "samcli/commands/local/lib/local_lambda.py", line 144, in invoke
  File "samcli/lib/telemetry/metric.py", line 324, in wrapped_func
  File "samcli/local/lambdafn/runtime.py", line 192, in invoke
  File "samcli/local/lambdafn/runtime.py", line 107, in create
  File "samcli/local/docker/manager.py", line 96, in create
  File "samcli/local/docker/container.py", line 167, in create
  File "samcli/local/docker/container.py", line 250, in _create_mapped_symlink_files

An unexpected error was encountered while executing "sam local invoke".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20local%20invoke%20-
%20FileNotFoundError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20local%20
invoke%20-%20FileNotFoundError
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
root@docker-desktop:/app/lambdas/feature/helloWorldLambda/dist#