Media 파일들을 S3에 저장하기
Django의 Media 파일들을 S3에 옮겨봅시다.
기존에 만든 블로그는 포스팅에 사용하는 이미지들이 로컬 웹 서버에 저장되고 있었습니다.
이미지가 늘어나면 EC2의 저장용량 문제가 생기는데,
S3를 사용하는 편이 비용 면에서나, 후에 트래픽 관리면에서나 좋을 것 같아 사용해보기로 하였습니다.
1. AWS S3 버킷 만들기
AWS 문서에 따르면,
버킷은 Amazon S3에 저장된 객체에 대한 컨테이너입니다
S3에 객체를 저장하려면, 버킷을 새로 만들어야 하는데요,
AWS에 로그인한 뒤, S3 서비스로 들어갑니다.
좌측 메뉴바에서 버킷을 선택한 뒤,
주황색 버킷 만들기 버튼을 선택합니다.
일반 설정 부분입니다.
원하는 버킷 이름을 짓고, S3를 사용할 리전(지역)도 선택합니다.
객체 소유권 설정 부분입니다.
객체 별로 소유권을 설정해서, 같은 버킷 내에 있는 객체이지만, ACL 설정을 통해 특정 AWS 계정만 접근할 수 있는 권한을 줄 수 있다고 합니다.
후에 Django 프로젝트 설정에서 Default ACL 설정을 통해 객체들에게 'public-read' 권한을 줄 것 이므로, 활성화 버튼을 눌러줍니다.
객체와 버킷에 대한 퍼블릭 엑세스 차단 설정 부분입니다.
이 부분은 S3를 사용하는 상황에 따라 다르게 설정해야 한다고 하는데,
저는 S3에 블로그에 들어갈 이미지들만 저장할 것 이므로,
체크를 해제해 퍼블릭 엑세스를 허용 한뒤, 진행하겠습니다.
나머지 설정은 기본 설정으로 두고,
버킷을 생성합니다.
2. S3에 접근할 유저 추가하기
S3에 접근 가능한 유저를 생성하여,
해당 유저의 Id와 Secret Key값을 Django에 추가하여 사용할 겁니다.
AWS에서, IAM 서비스로 들어갑니다.
좌측 탭에서 사용자를 선택하고,
파란색 사용자 추가 버튼을 선택합니다.
사용자 이름을 작성하고,
하단에서는 액세스 키 - 프로그래밍 방식 액세스를 선택합니다.
저희는 이 사용자를 S3에 접근 가능한 그룹을 생성하고,
이 그룹안에 넣어줄 겁니다.
이어 나오는 창에서 그룹 생성 버튼을 클릭합니다.
그룹 이름을 작성하고,
하단에서는 s3full을 검색한 뒤, 나오는 AmazonS3FullAccess를 체크하고, 그룹을 생성합니다.
이후 사용자 생성 과정은 별도의 설정 없이,
기본 설정으로 진행하겠습니다.
이어 나오는 액세스 키와 비밀 엑세스 키는,
후에 다시 확인하는 것이 불가하므로 .csv 파일을 받아놓습니다.
3. Django 프로젝트 설정
pip를 통해 boto3와 django-storages를 설치합니다.
$ pip install boto3
$ pip install django-storages
프로젝트의 settings.py파일을 수정합니다.
INSTALLED_APPS에 storages를 추가합니다.
INSTALLED_APPS = [
...
'storages',
]
settings.py 하단에 아래 내용을 추가합니다.
# S3 Storages
AWS_ACCESS_KEY_ID = '.csv 파일에 있는 Access Key Id'
AWS_SECRET_ACCESS_KEY = '.csv 파일에 있는 Secret Access Key'
AWS_REGION = 'ap-northeast-2'
AWS_STORAGE_BUCKET_NAME = 'codecamper-blog-bucket'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.%s.amazonaws.com' % (AWS_STORAGE_BUCKET_NAME, AWS_REGION)
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
현재 제 블로그는 nginx를 사용하고 있으므로, nginx.conf 파일도 수정합니다.
/media/ 경로로 요청이 들어오면 기존에는 alias로 로컬 디렉토리를 가리키고 있었지만,
이제는 S3 서버로 요청을 보내 응답받아 이미지를 돌려줘야 하므로, proxy_pass를 통해 S3 경로르 적어줍니다.
server {
...
location /media/ {
client_max_body_size 10M;
proxy_pass https://codecamper-blog-bucket.s3.ap-northeast-2.amazonaws.com/;
}
}
위 작업까지 마치고 runserver를 통해 실행한 뒤,
admin 서버를 통해 파일을 올려보면 정상적으로 AWS 콘솔에서 S3에 파일이 올라간 것을 확인할 수 있습니다.
3. 번외 - 로컬에 있던 image들을 S3로 옮기기
저는 이전에도 글들을 포스팅하며 사용했던 이미지들은 이미 웹 서버의 내부 저장소에 있는 상태입니다.
이 이미지들을 custom command를 만들어,
그 command로 S3로 옮겨보겠습니다.
프로젝트 폴더의 앱 폴더 내에 /management/commands 디렉토리를 생성하고,
해당 디렉토리 내에 migrate_media_files_to_s3.py 라는 새 파일을 하나 생성한 뒤, 아래 코드를 붙여넣습니다.
from django.core.management.base import BaseCommand
from django.conf import settings
import logging
import os
import boto3
from botocore.exceptions import ClientError
class Command(BaseCommand):
def handle(self, *args, **options):
# create session to s3
session = boto3.session.Session()
s3_client = session.client(
service_name='s3',
aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY
)
# process migrate local media files to s3
for full_path in self._get_all_media_file():
try:
upload_path = self._split_path_to_upload(full_path)
args = dict()
args["ACL"] = 'public-read'
s3_client.upload_file(full_path, settings.AWS_STORAGE_BUCKET_NAME, upload_path, args)
print(f"success upload {upload_path}")
logging.info(f"success upload {upload_path}")
except ClientError as e:
print(f"failed upload {upload_path}")
logging.error(f"{e}: {upload_path}")
def _get_all_media_file(self) -> [str]:
"""
this method is used to retrieve all media files contained in this project.
:return: list of string path file
"""
files = []
for r, d, f in os.walk(settings.MEDIA_ROOT):
for file in f:
files.append(os.path.join(r, file))
return files
def _split_path_to_upload(self, full_path: str) -> str:
"""
This function is used to get the string media path
:param full_path: string path of file
:return: string media path to upload s3
"""
media_root = settings.MEDIA_ROOT
upload_path = full_path.split(media_root)[-1]
upload_path = self._remove_leading_slash(upload_path)
return upload_path
def _remove_leading_slash(self, path: str) -> str:
"""
저의 기존 media_root가 'media'로, 뒤에 붙는 '/'가 없었기 때문에,
split 후 path 제일 앞에 /가 붙어 이를 없애는 함수입니다.
사용하시는 분에 따라 삭제하셔도 되는 함수입니다.
"""
if path[0] == '/':
return path[1:]
else:
return path
그런 다음, 쉘에서 python manage.py migrate_media_files_to_s3로 실행하면,
로컬에 있던 이미지들이 S3로 업로드됩니다.
Привет господа!
http://forum.soundspeed.ru/member.php?588973-Marinabux
http://htcclub.pl/member.php/279058-Marinahjh
http://forum.soundspeed.ru/member.php?589484-Marinaasl
http://147.91.246.89/viewtopic.php?f=31&t=15121
http://arabfm.net/vb/showthread.php?p=3875976#post3875976
Предлагаем Вашему вниманию сервис оргтехники и продажу расходников для дома и офиса.Наша организация «КОПИМЕДИАГРУПП» работает 10 лет на рынке этой продукции в Беларуси.В сервисном центре клиенты могут заправить картридж сертифицированным совместимым тонером. Мастера проводят перезаправку картриджей лазерных устройств и струйных со встроенной СНПЧ. В случае необходимости можно сделать перепрошивку или замену чипа, благодаря которым пользователи смогут перезаправлять новые чернила практически неограниченное количество раз.Заправка картриджей для принтера обязательно включает проверку состояния комплектующих, в числе которых фотобарабан, вал первичного заряда, ракель, магнитный вал и другие запчасти. При необходимости неисправные или изношенные детали заменяются на новые. На замену стандартным – специалисты могут установить перезаправляемые картриджи.В список предоставляемых услуг входит ремонт и восстановление техники:лазерных и струйных принтеров;МФУ;копировальных аппаратов;факсов;плоттеров.Высокая квалификация специалистов и наличие профильного инструмента позволяют проводить точную диагностику и устранять неисправности абсолютно любого характера.Для устройств Samsung и Xerox проводится перепрошивка. Установка нового программного обеспечения снимает блокировку на заправку оригинальных и установку совместимых картриджей.Как заказать заправку картриджей
Thanks for the post
Enjoy a Luxurious Vacation in Bali
An impressive share! I have just forwarded this onto a co-worker who was doing a little research on this. And he actually bought me dinner due to the fact that I discovered it for him... lol. So allow me to reword this.... Thank YOU for the meal!! But yeah, thanks for spending time to talk about this matter here on your web site.
Luxury Bali Villas
thanks, interesting read
Find Your Dream Villa in Bali
What a data of un-ambiguity and preserveness of precious familiarity about unpredicted emotions.
Find your dream Bali villa
Hello there! Quick question that's entirely off topic. Do you know how to make your site mobile friendly? My weblog looks weird when browsing from my iphone. I'm trying to find a theme or plugin that might be able to fix this problem. If you have any recommendations, please share. Appreciate it!
I enjoy reading through a post that will make people think. Also, thank you for permitting me to comment!
Excellent web site. A lot of useful information here. I'm sending it to a few buddies ans additionally sharing in delicious. And certainly, thanks for your effort!
Hi there to all, it's genuinely a pleasant for me to visit this website, it includes helpful Information.
It's hard to find educated people on this subject, but you sound like you know what you're talking about! Thanks
Good replies in return of this query with genuine arguments and explaining everything regarding that.
After exploring a few of the blog articles on your site, I honestly like your way of blogging. I saved as a favorite it to my bookmark webpage list and will be checking back in the near future. Take a look at my web site as well and let me know how you feel.
I've read a few good stuff here. Definitely price bookmarking for revisiting. I surprise how so much effort you put to create this sort of great informative site.
Excellent site. Plenty of useful info here. I'm sending it to a few buddies ans also sharing in delicious. And certainly, thank you for your effort!
It's genuinely very complex in this busy life to listen news on Television, so I just use world wide web for that reason, and get the most up-to-date information.
Excellent web site you have got here.. It's hard to find quality writing like yours these days. I really appreciate people like you! Take care!!
Good article! We are linking to this great article on our site. Keep up the great writing.
There's certainly a lot to know about this issue. I love all the points you've made.
I used to be suggested this website by means of my cousin. I'm now not positive whether or not this publish is written by him as no one else recognize such specified about my difficulty. You're amazing! Thanks!
Thank you for any other informative blog. Where else may I am getting that kind of info written in such a perfect means? I have a mission that I am just now operating on, and I've been at the look out for such information.
I have read so many posts on the topic of the blogger lovers but this piece of writing is truly a pleasant paragraph, keep it up.
In my opinion you are not right. I can prove it. Write to me in PM, we will communicate.
notoriety
[url=https://domkink.com/adult/adult-truth-or-dare-pornopat-22.php]domkink.com[/url]
@7567s3gQ
Hello, after reading this amazing post i am as well cheerful to share my know-how here with colleagues.
Hi there excellent website! Does running a blog like this require a massive amount work? I've absolutely no knowledge of programming but I was hoping to start my own blog in the near future. Anyways, if you have any ideas or techniques for new blog owners please share. I know this is off topic nevertheless I just had to ask. Many thanks!
This post will assist the internet people for creating new webpage or even a weblog from start to end.
Valuable information. Fortunate me I discovered your website unintentionally, and I'm shocked why this accident didn't happened in advance! I bookmarked it.
I got this site from my friend who shared with me about this web site and now this time I am visiting this website and reading very informative posts at this place.
Very good site you have here but I was curious about if you knew of any user discussion forums that cover the same topics talked about in this article? I'd really love to be a part of group where I can get opinions from other experienced individuals that share the same interest. If you have any suggestions, please let me know. Bless you!
Your style is very unique compared to other people I've read stuff from. Thank you for posting when you have the opportunity, Guess I'll just book mark this site.
I'm amazed, I have to admit. Rarely do I encounter a blog that's both educative and engaging, and without a doubt, you have hit the nail on the head. The issue is something that too few men and women are speaking intelligently about. I am very happy I stumbled across this during my hunt for something concerning this.
Hmm is anyone else encountering problems with the images on this blog loading? I'm trying to determine if its a problem on my end or if it's the blog. Any feed-back would be greatly appreciated.
With havin so much content do you ever run into any issues of plagorism or copyright violation? My site has a lot of unique content I've either created myself or outsourced but it seems a lot of it is popping it up all over the web without my authorization. Do you know any ways to help protect against content from being stolen? I'd really appreciate it.
Hi everyone, it's my first go to see at this site, and paragraph is truly fruitful designed for me, keep up posting these articles.
Remarkable! Its truly awesome post, I have got much clear idea regarding from this article.
This is my first time go to see at here and i am genuinely impressed to read all at single place.
Very rapidly this web site will be famous amid all blogging and site-building viewers, due to it's good posts
Please let me know if you're looking for a article writer for your site. You have some really great posts and I believe I would be a good asset. If you ever want to take some of the load off, I'd really like to write some material for your blog in exchange for a link back to mine. Please blast me an email if interested. Regards!
My family members every time say that I am killing my time here at net, but I know I am getting familiarity daily by reading such good articles.
Hello every one, here every one is sharing these experience, therefore it's nice to read this weblog, and I used to go to see this blog everyday.
This is my first time go to see at here and i am actually impressed to read all at single place.
Helpful info. Lucky me I discovered your web site unintentionally, and I am surprised why this coincidence did not took place earlier! I bookmarked it.
Hey there, I think your blog might be having browser compatibility issues. When I look at your blog site in Opera, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, awesome blog!
If some one wants expert view concerning blogging after that i propose him/her to pay a quick visit this weblog, Keep up the pleasant job.
Thanks for a marvelous posting! I really enjoyed reading it, you could be a great author.I will be sure to bookmark your blog and will come back in the foreseeable future. I want to encourage yourself to continue your great work, have a nice afternoon!
Hello Dear, are you truly visiting this website regularly, if so after that you will definitely get nice experience.
I just like the helpful information you provide in your articles. I'll bookmark your blog and test once more here regularly. I am fairly certain I'll be told many new stuff right here! Good luck for the next!
I think that what you composed made a ton of sense. But, think on this, what if you typed a catchier post title? I am not saying your content is not solid., however what if you added something that grabbed a person's attention? I mean %BLOG_TITLE% is a little plain. You could peek at Yahoo's home page and watch how they write post headlines to grab people to open the links. You might add a related video or a related picture or two to get readers excited about what you've got to say. Just my opinion, it could bring your posts a little bit more interesting.
Fine way of explaining, and nice article to take facts on the topic of my presentation focus, which i am going to present in institution of higher education.
Hey! Quick question that's totally off topic. Do you know how to make your site mobile friendly? My website looks weird when viewing from my iphone. I'm trying to find a theme or plugin that might be able to correct this issue. If you have any recommendations, please share. Many thanks!
It's awesome designed for me to have a website, which is good in support of my experience. thanks admin
For most recent news you have to pay a quick visit world-wide-web and on internet I found this web page as a most excellent web site for hottest updates.
If you are going for best contents like myself, only pay a quick visit this website daily for the reason that it presents quality contents, thanks
Wonderful, what a website it is! This website gives helpful facts to us, keep it up.
Hi, I wish for to subscribe for this weblog to take newest updates, so where can i do it please assist.
Thanks , I've just been searching for info approximately this topic for a long time and yours is the best I've came upon till now. But, what about the conclusion? Are you positive about the source?
If some one wishes expert view on the topic of blogging and site-building afterward i recommend him/her to pay a visit this web site, Keep up the pleasant work.
Great goods from you, man. I've be aware your stuff prior to and you are simply too wonderful. I actually like what you have acquired right here, certainly like what you're saying and the best way during which you say it. You're making it entertaining and you still take care of to keep it sensible. I cant wait to read far more from you. This is really a terrific site.
Hi there everybody, here every person is sharing these experience, therefore it's nice to read this webpage, and I used to pay a visit this website all the time.
I don't even know how I ended up here, however I believed this submit was once great. I do not realize who you are but certainly you are going to a famous blogger in case you are not already. Cheers!
I don't know if it's just me or if everyone else experiencing problems with your website. It appears like some of the text within your content are running off the screen. Can somebody else please comment and let me know if this is happening to them as well? This might be a problem with my internet browser because I've had this happen before. Many thanks
Hi everybody, here every one is sharing these experience, so it's nice to read this weblog, and I used to visit this blog daily.
Hi there, everything is going fine here and ofcourse every one is sharing facts, that's genuinely excellent, keep up writing.