5.4. AWS Bedrock API 설정 (Claude, Mistral, LLaMA, ...)

AWS Bedrock 에서 제공하는 LLM 을 AWS 에서 사용하도록 설정하는 방법
Antropoic Claude, Mistral AI, Meta LLaMA 등 이용 가능

AWS 콘솔 로그인

http://console.aws.amazon.com

IAM 서비스 방문 및 사용자 추가

http://console.aws.amazon.com/iam/home

이름	원하는 대로
그룹	원하는 대로
권한 정책	AmazonBedrockFullAccess

API 키 생성

⭐ API 키는 잊어버리지 않도록 잘 관리해야 함

.env 파일 안의 두 변수(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)에 넣어야 하는 값이 나옴

모델 사용 신청

https://console.aws.amazon.com/bedrock/home

BedRock 서비스 → 리전 선택 → Model catalog → 원하는 Serverless 모델 선택 → 요청

리전에 따라 이용가능한 모델이 다르니 참고할 것

litellm_config.yml 수정

os.environ/XXXX 은 .env 파일에 선언된 XXXX 변수의 값을 사용하겠다는 의미

상세한 옵션은 다음 문서를 참고할 것
https://docs.litellm.ai/docs/providers/bedrock

참고로 model_name 이 동일한 것이 여러개 선언한 것은 LiteLLM 의 Load Balance 기능으로 API 부하를 분산함

litellm_config.yml

model_list:

  - model_name: claude-3-haiku
    litellm_params:
      model: bedrock/apac.anthropic.claude-3-haiku-20240307-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-northeast-2"

  - model_name: claude-3-haiku
    litellm_params:
      model: bedrock/us.anthropic.claude-3-haiku-20240307-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: claude-3.5-haiku
    litellm_params:
      model: bedrock/us.anthropic.claude-3-5-haiku-20241022-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: claude-3-sonnet
    litellm_params:
      model: bedrock/apac.anthropic.claude-3-sonnet-20240229-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-northeast-2"

  - model_name: claude-3-sonnet
    litellm_params:
      model: bedrock/us.anthropic.claude-3-sonnet-20240229-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: claude-3.5-sonnet
    litellm_params:
      model: bedrock/apac.anthropic.claude-3-5-sonnet-20240620-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-northeast-2"

  - model_name: claude-3.5-sonnet
    litellm_params:
      model: bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: claude-3.5-sonnet-v2
    litellm_params:
      model: bedrock/us.anthropic.claude-3-5-sonnet-20241022-v2:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: claude-3-opus
    litellm_params:
      model: bedrock/us.anthropic.claude-3-opus-20240229-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: mistral-small-24.02
    litellm_params:
      model: bedrock/mistral.mistral-small-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-east-1"

  - model_name: mistral-small-24.02
    litellm_params:
      model: bedrock/mistral.mistral-small-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: mistral-small-24.02
    litellm_params:
      model: bedrock/mistral.mistral-small-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-southeast-2"

  - model_name: mistral-7b-instruct
    litellm_params:
      model: bedrock/mistral.mistral-7b-instruct-v0:2
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-east-1"

  - model_name: mistral-7b-instruct
    litellm_params:
      model: bedrock/mistral.mistral-7b-instruct-v0:2
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: mistral-7b-instruct
    litellm_params:
      model: bedrock/mistral.mistral-7b-instruct-v0:2
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-southeast-2"

  - model_name: mistral-large-24.02
    litellm_params:
      model: bedrock/mistral.mistral-large-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-east-1"

  - model_name: mistral-large-24.02
    litellm_params:
      model: bedrock/mistral.mistral-large-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: mistral-large-24.02
    litellm_params:
      model: bedrock/mistral.mistral-large-2402-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "ap-southeast-2"

  - model_name: llama3.2-1b-instruct
    litellm_params:
      model: bedrock/us.meta.llama3-2-1b-instruct-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: llama3.2-3b-instruct
    litellm_params:
      model: bedrock/us.meta.llama3-2-3b-instruct-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: llama3.2-11b-instruct
    litellm_params:
      model: bedrock/us.meta.llama3-2-11b-instruct-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"

  - model_name: llama3.2-90b-instruct
    litellm_params:
      model: bedrock/us.meta.llama3-2-90b-instruct-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: "us-west-2"