S3 API Compatibility support

[M Ranga Swami Reddy]

Hi Team: As per the ceph document a few S3 APIs compatibility not suppo=
rted.

Link: http://ceph.com/docs/master/radosgw/s3/

Is there plan to support the =C3=BCn supported item in the above table?
or
Any working on this?

Thanks
Swami
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" i=
n
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Sage Weil]

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.

Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...

sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Yehuda Sadeh]

On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.

Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.

Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Can you please share the details on coverage of the ceph/s3-tests suite?
(mean how many S3 compatibility APIs tests performed with s3-tests suite)

Thanks
Swami

On Sun, Jul 27, 2014 at 1:24 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Yehuda Sadeh]

There's no good enumeration of the actual apis and features that are
covered by this test. It's definitely a worthy cause to pursue though,
and one that could be done with the help of a willing community
member. The s3 test suite can be found here:

https://github.com/ceph/s3-tests

Thanks,
Yehuda

On Mon, Jul 28, 2014 at 6:05 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Can you please share the details on coverage of the ceph/s3-tests suite?
(mean how many S3 compatibility APIs tests performed with s3-tests suite)
Thanks
Swami
On Sun, Jul 27, 2014 at 1:24 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Yehuda Sadeh]

Bucket lifecycle:
http://tracker.ceph.com/issues/8929

Bucket notification:
http://tracker.ceph.com/issues/8956

On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi Yehuda,

Could you please check and clarify the below question on object
lifecycle and notification S3 APIs support:

1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
For ex: If an object lifecyle set as below:
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.

Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?

2. To support the object notifications:
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.

Q2- Does Ceph support low cost and high availability storage type?

Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi ,

Could you please check and clarify the below question on object
lifecycle and notification S3 APIs support:

1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
For ex: If an object lifecyle set as below:
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.

Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?

2. To support the object notifications:
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.

Q2- Does Ceph support low cost and high availability storage type?

Thanks

On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Thanks
Swami

On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Sage Weil]

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.

For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi Sage,
Thanks for quick reply.

Post by Sage Weil
Ceph doesn't interact at all with AWS services like Glacier, if that's

No. I meant that - Ceph interaction with a glacier and RRS type of
storages along with currently used OSD (or standard storage).

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.

Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Sage Weil]

Post by M Ranga Swami Reddy
Hi Sage,
Thanks for quick reply.

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

What do you mean by "RRS storage-low cost storage"? My read of the RRS
numbers is that they simply have a different tier of S3 that runs fewer
replicas and (probably) cheaper disks. In radosgw-land, this would just
be a different rados pool with 2x replicas and (probably) a CRUSH rule
mapping it to different hardware (with bigger and/or cheaper disks).

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to
S3.

Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Post by M Ranga Swami Reddy

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.

Great to hear!

Thanks-
sage

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Post by Sage Weil
What do you mean by "RRS storage-low cost storage"? My read of the RRS
numbers is that they simply have a different tier of S3 that runs fewer
replicas and (probably) cheaper disks. In radosgw-land, this would just
be a different rados pool with 2x replicas and (probably) a CRUSH rule
mapping it to different hardware (with bigger and/or cheaper disks).

Thats correct. If we could do the with a different rados pool using
2x replicas along with CURSH
mapping it to different h/w (with bigger and cheaper disks) , then its
same as RRS support in AWS.

Post by Sage Weil

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to S3.
Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Sure, I will create a new feature ticket and add the needful information there.

Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Thanks for quick reply.

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

What do you mean by "RRS storage-low cost storage"? My read of the RRS
numbers is that they simply have a different tier of S3 that runs fewer
replicas and (probably) cheaper disks. In radosgw-land, this would just
be a different rados pool with 2x replicas and (probably) a CRUSH rule
mapping it to different hardware (with bigger and/or cheaper disks).

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to
S3.
Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Post by M Ranga Swami Reddy

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.

Great to hear!
Thanks-
sage

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Post by M Ranga Swami Reddy

Post by Sage Weil
There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to S3.
Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Sure, I will create a new feature ticket and add the needful information there.

Created a new ticket: http://tracker.ceph.com/issues/9581


Thanks
Swami

On Fri, Sep 19, 2014 at 9:23 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy

Post by Sage Weil
What do you mean by "RRS storage-low cost storage"? My read of the RRS
numbers is that they simply have a different tier of S3 that runs fewer
replicas and (probably) cheaper disks. In radosgw-land, this would just
be a different rados pool with 2x replicas and (probably) a CRUSH rule
mapping it to different hardware (with bigger and/or cheaper disks).

Thats correct. If we could do the with a different rados pool using
2x replicas along with CURSH
mapping it to different h/w (with bigger and cheaper disks) , then its
same as RRS support in AWS.

Post by Sage Weil

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to S3.
Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Sure, I will create a new feature ticket and add the needful information there.
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Thanks for quick reply.

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

What do you mean by "RRS storage-low cost storage"? My read of the RRS
numbers is that they simply have a different tier of S3 that runs fewer
replicas and (probably) cheaper disks. In radosgw-land, this would just
be a different rados pool with 2x replicas and (probably) a CRUSH rule
mapping it to different hardware (with bigger and/or cheaper disks).

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

There is the main #4099 issue for object expiration, but there is no real
detail there. The plan is (as always) to have equivalent functionality to S3.
Do you mind creating a new feature ticket that specifically references the
ability to move objects to a second storage tier based on policy? Any
references to AWS docs about the API or functionality would be helpful in
the ticket.

Post by M Ranga Swami Reddy

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.

Great to hear!
Thanks-
sage

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Yehuda Sadeh]

On Fri, Sep 19, 2014 at 8:32 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Sage,
Thanks for quick reply.

Post by Sage Weil
Ceph doesn't interact at all with AWS services like Glacier, if that's

No. I meant that - Ceph interaction with a glacier and RRS type of
storages along with currently used OSD (or standard storage).

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

You can use multiple placement targets and can specify on bucket
creation which placement target to use. At this time we don't support
the exact S3 reduced redundancy fields, although it should be pretty
easy to add.

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

I think #8929 would cover it.

Yehuda

Post by M Ranga Swami Reddy

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Post by Yehuda Sadeh

Post by M Ranga Swami Reddy
Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

You can use multiple placement targets and can specify on bucket
creation which placement target to use. At this time we don't support
the exact S3 reduced redundancy fields, although it should be pretty
easy to add.

OK. Could you please guide us to implement the above.

Post by Yehuda Sadeh

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

I think #8929 would cover it.

The above issue for bucket lifecyle, but we are storing the
object/buckets on a separate
disk like RRS type. I think, its needed to support an object to be
stored based on the
option (default on standard storage).


Thanks
Swami

Post by Yehuda Sadeh
On Fri, Sep 19, 2014 at 8:32 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Sage,
Thanks for quick reply.

Post by Sage Weil
Ceph doesn't interact at all with AWS services like Glacier, if that's

No. I meant that - Ceph interaction with a glacier and RRS type of
storages along with currently used OSD (or standard storage).

Post by Sage Weil
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.

Yes, we can confiure ceph to use 2x replicas, which will look like
reduced redundancy, but AWS uses a separate RRS storage-low cost
(instead of
standard) storage for this purpose. I am checking, if we could
similarly in ceph too.

You can use multiple placement targets and can specify on bucket
creation which placement target to use. At this time we don't support
the exact S3 reduced redundancy fields, although it should be pretty
easy to add.

Post by M Ranga Swami Reddy

Post by Sage Weil
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.

OK. Do we have the issue id for the above? Else, we can file one. Please advise.

I think #8929 would cover it.
Yehuda

Post by M Ranga Swami Reddy

Post by Sage Weil
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...

Great to know this. Even we are keen with S3 support in Ceph and we
are happy support you here.
Thanks
Swami

Post by Sage Weil

Post by M Ranga Swami Reddy
Hi Sage,
Could you please advise, if Ceph support the low cost object
storages(like Amazon Glacier or RRS) for archiving objects like log
file etc.?

Ceph doesn't interact at all with AWS services like Glacier, if that's
what you mean.
For RRS, though, I assume you mean the ability to create buckets with
reduced redundancy with radosgw? That is supported, although not quite
the way AWS does it. You can create different pools that back RGW
buckets, and each bucket is stored in one of those pools. So you could
make one of them 2x instead of 3x, or use an erasure code of your choice.
What isn't currently supported is the ability to reduce the redundancy of
individual objects in a bucket. I don't think there is anything
architecturally preventing that, but it is not implemented or supported.
When we look at the S3 archival features in more detail (soon!) I'm sure
this will come up! The current plan is to address object versioning
first. That is, unless a developer surfaces who wants to start hacking on
this right away...
sage

Post by M Ranga Swami Reddy
Thanks
Swami
On Thu, Sep 18, 2014 at 6:20 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi ,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
On Fri, Sep 12, 2014 at 8:00 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Could you please check and clarify the below question on object
1. To support the bucket lifecycle - we need to support the
moving/deleting the objects/buckets based lifecycle settings.
1. Archive it after 10 days - means move this object to low
cost object storage after 10 days of the creation date.
2. Remove this object after 90days - mean remove this
object from the low cost object after 90days of creation date.
Q1- Does the ceph support the above concept like moving to low cost
storage and delete from that storage?
- First there should be low cost and high availability storage
with single replica only. If an object created with this type of
object storage,
There could be chances that object could lose, so if an object
of this type of storage lost, set the notifications.
Q2- Does Ceph support low cost and high availability storage type?
Thanks
Swami

Post by Yehuda Sadeh
http://tracker.ceph.com/issues/8929
http://tracker.ceph.com/issues/8956
On Sun, Jul 27, 2014 at 12:54 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Good no know the details. Can you please share the issue ID for bucket
lifecycle? My team also could start help here.
Regarding the notification - Do we have the issue ID?
Yes, the object versioning will be backlog one - I strongly feel we
start working on this asap.
Thanks
Swami

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi Yehuda,
I tried to test the get bucket location API, but got an error as
below. Is this known issue? Could you please confirm. Thanks,Swami
====
In [14]: b.get_location()
send: 'GET /some-bucket-from-libs3/?
location HTTP/1.1\r\nHost: 192.168.122.61\r\nAccept-Encoding:
identity\r\nDate: Fri, 01 Aug 2014 12:02:22 GMT\r\nContent-Length:
0\r\nAuthorization: AWS
cec157b52253463b91547e0a73369960:sT5aw9yXantfAafRn/6x+IgI1pk=\r\nUser-Agent:
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: ''
send: 'GET /some-bucket-from-libs3/?location HTTP/1.1\r\nHost:
192.168.122.61\r\nAccept-Encoding: identity\r\nDate: Fri, 01 Aug 2014
12:02:22 GMT\r\nContent-Length: 0\r\nAuthorization: AWS
cec157b52253463b91547e0a73369960:sT5aw9yXantfAafRn/6x+IgI1pk=\r\nUser-Agent:
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Fri, 01 Aug 2014 12:02:23 GMT
header: Server: Apache/2.4.7 (Ubuntu)
header: Vary: Accept-Encoding
header: Transfer-Encoding: chunked
header: Content-Type: application/xml
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-14-7c597523523d> in <module>()
----> 1 b.get_location()

/usr/lib/python2.7/dist-packages/boto/s3/bucket.pyc in get_location(self)
1005 h = handler.XmlHandler(rs, self)
1006 xml.sax.parseString(body, h)
-> 1007 return rs.LocationConstraint
1008 else:
1009 raise self.connection.provider.storage_response_error(

AttributeError: 'ResultSet' object has no attribute 'LocationConstraint'
===

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Hi Yehuda,
Can I please get any suggestion on the "get bucket_location" API
error? Please share any info, which can help us to debug more here.

Thanks
Swami

On Mon, Aug 4, 2014 at 4:44 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
I tried to test the get bucket location API, but got an error as
below. Is this known issue? Could you please confirm. Thanks,Swami
====
In [14]: b.get_location()
send: 'GET /some-bucket-from-libs3/?
0\r\nAuthorization: AWS
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: ''
192.168.122.61\r\nAccept-Encoding: identity\r\nDate: Fri, 01 Aug 2014
12:02:22 GMT\r\nContent-Length: 0\r\nAuthorization: AWS
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Fri, 01 Aug 2014 12:02:23 GMT
header: Server: Apache/2.4.7 (Ubuntu)
header: Vary: Accept-Encoding
header: Transfer-Encoding: chunked
header: Content-Type: application/xml
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-14-7c597523523d> in <module>()
----> 1 b.get_location()
/usr/lib/python2.7/dist-packages/boto/s3/bucket.pyc in get_location(self)
1005 h = handler.XmlHandler(rs, self)
1006 xml.sax.parseString(body, h)
-> 1007 return rs.LocationConstraint
1009 raise self.connection.provider.storage_response_error(
AttributeError: 'ResultSet' object has no attribute 'LocationConstraint'
===

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Yehuda Sadeh]

At the moment we don't support this api call. Adding it should be
pretty easy, and it's definitely something that could be done with the
help of a willing community member.

Yehuda

On Tue, Aug 12, 2014 at 9:58 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
Can I please get any suggestion on the "get bucket_location" API
error? Please share any info, which can help us to debug more here.
Thanks
Swami
On Mon, Aug 4, 2014 at 4:44 PM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Hi Yehuda,
I tried to test the get bucket location API, but got an error as
below. Is this known issue? Could you please confirm. Thanks,Swami
====
In [14]: b.get_location()
send: 'GET /some-bucket-from-libs3/?
0\r\nAuthorization: AWS
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: ''
192.168.122.61\r\nAccept-Encoding: identity\r\nDate: Fri, 01 Aug 2014
12:02:22 GMT\r\nContent-Length: 0\r\nAuthorization: AWS
Boto/2.20.1 Python/2.7.6 Linux/3.13.0-29-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Fri, 01 Aug 2014 12:02:23 GMT
header: Server: Apache/2.4.7 (Ubuntu)
header: Vary: Accept-Encoding
header: Transfer-Encoding: chunked
header: Content-Type: application/xml
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-14-7c597523523d> in <module>()
----> 1 b.get_location()
/usr/lib/python2.7/dist-packages/boto/s3/bucket.pyc in get_location(self)
1005 h = handler.XmlHandler(rs, self)
1006 xml.sax.parseString(body, h)
-> 1007 return rs.LocationConstraint
1009 raise self.connection.provider.storage_response_error(
AttributeError: 'ResultSet' object has no attribute 'LocationConstraint'
===

Post by Yehuda Sadeh
On Fri, Jul 25, 2014 at 10:14 AM, M Ranga Swami Reddy

Post by M Ranga Swami Reddy
Thanks for quick reply.
Yes, versioned object - missing in ceph ATM
Iam looking for: bucket lifecylce (get/put/delete), bucket location,
put object notification and object restore (ie versioned object) S3
API support.
Please let me now any of the above work is in progress or some one
planned to work on.

I opened an issue for bucket lifecycle (we already had an issue open
for object expiration though). We do have bucket location already
(part of the multi-region feature). Object versioning is definitely on
our backlog and one that we'll hopefully implement sooner rather
later.
With regard to object notification, it'll require having a
notification service which is a bit out of the scope. Integrating the
gateway with such a service whouldn't be hard, but we'll need to have
that first.
Yehuda

Post by M Ranga Swami Reddy
Thanks
Swami

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[M Ranga Swami Reddy]

Thanks Yehuda. I am testing the S3 APIs suing the openstack+ceph.
Could able test a couple of APIs like put/get/delete bucket (using the
boto python libraries). When I tried for get bucket location I got an
error mentioned in my previous email.

And explored the ceph source code and not able see the appropriate
code/function for "get_bucket_location". But as per the previous email
discuss,
the bucket_location supported as part multi-region support.
Let me know, if I miss something here.

Thanks
Swami
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Abhishek L]

Post by Yehuda Sadeh
At the moment we don't support this api call. Adding it should be
pretty easy, and it's definitely something that could be done with the
help of a willing community member.
Yehuda

Hi,

I have a basic implementation for S3 get bucket location, which returns
the region of the bucket.[1] For the "default" region it returns the
empty string similiar to s3 (which returns empty string for the
"default" us-east region).

Please let me know whether this is acceptable (or if I have missed
something obvious), and what else needs to be done in order to get this
in. Once I have a review on code I'll try to update the docs & examples
regarding this call.


[1] https://github.com/ceph/ceph/pull/2329

Thanks

[Robin H. Johnson]

Post by Sage Weil

Hi Team: As per the ceph document a few S3 APIs compatibility not supported.
Link: http://ceph.com/docs/master/radosgw/s3/
Is there plan to support the ?n supported item in the above table?
or
Any working on this?

Yes. Unfortunately this table isn't particularly detailed or accurate or
up to date. The main gap, I think, is versioned objects.
Are there specfiic parts of the S3 API that are missing that you need?
That sort of info is very helpful for prioritizing effort...

Fine-grained object policies would be useful here, w/ users+groups
brought in from keystone.