How to define and use a temporary url in Openstack?
Our temporary url needs and usages with Kotlin&Go examples
As Influencer Communications & Favorites Team, we lately have been struggling to create a temporary url in Openstack for influencers’ invoices.
We wanted to share our story and tell you why and how we did it.
Why did we use temporary url?
We were storing influencers’ invoices data in Openstack but we didn’t want it to be public (since it is personal data).
For this, we searched the possible solutions and we found that there is a feature in Openstack which is called Temporary URL Middleware.
This feature helps us to reach certain files that are actually in a private container, but helps us to reach it with a specific retention.
So this was our roadmap:
- Create a private container in Openstack. (Oss as short name)
- Store personal data in this container.
- Whenever someone tries to access a data without proper authentication, they will fail.
- Create a temporary (temp) url for the people who are eligible to read a specific data. For this purpose, use a secret key defined.
While implementing this, we had all the steps we needed up until the 4th step. So that was our main purpose when finalizing this task.
Implementation
"tempUrlSecretKey": "lorem-ipsum",
"tempUrlSecretKey2": "dolor-sit-amet"
"apiKey": "api--key-Mehmet-Emre"We were storing this data in a secret place, such as HashiCorp Vault, for more details you can refer to its address below:
This is our non-secret config:
oss-config:
id: 123
container: invoice-data-1
oss-path: "oss.example.com"
env: production
ttl-in-minutes: 5Using our apiKey and id, we request to oss’s auth endpoint to get an authentication token for our app and refresh it with a specific interval.
We’ll start the full flow with Kotlin, and after it is finished, we’ll also add a Go example.
This is our full flow in Kotlin, starting with client code:
@Path("/v3/auth/tokens")
@RegisterRestClient(configKey = "oss-auth")
interface OssAuthClient {
@GZIP
@POST
@Produces("application/json")
@Consumes("application/json")
@Timeout(1000)
@Retry()
fun fetchToken(@RequestBody request: String): Response
}This is our service layer for auth:
@Singleton
class OssAuthService @Inject constructor(
@RestClient private val ossAuthClient: OssAuthClient,
private val ossConfiguration: OssConfiguration
) {
private var token: String? = null
@Scheduled(every = "36000s")
fun refreshToken() {
token = fetchToken()
}
fun getToken(): String? {
if (token == null) {
token = fetchToken()
}
return token
}
fun fetchToken(): String {
val tokenRequest: String = buildTokenRequest(ossConfiguration.id(),
ossConfiguration.secret())
val response: Response = ossAuthClient.fetchToken(tokenRequest)
val headers: MultivaluedMap<String, Any>? = response.headers
val tokenResponse: MutableList<Any>? = headers?.get("x-subject-token")
if (tokenResponse == null || tokenResponse.isEmpty()) {
throw ExampleException("Error while fetching token")
}
return tokenResponse[0].toString()
}
fun buildTokenRequest(id: String, secret: String): String {
return "{\n" +
" \"auth\":{\n" +
" \"identity\":{\n" +
" \"methods\":[\n" +
" \"application_credential\"\n" +
" ],\n" +
" \"application_credential\":{\n" +
" \"id\":\"" + id + "\",\n" +
" \"secret\":\"" + secret + "\"\n" +
" }\n" +
" }\n" +
" }\n" +
"}"
}
}This is our normal oss client code:
@GZIP
@POST
@Consumes("application/json")
@Produces("application/json")
@Timeout(10000)
@Retry()
@Path("/")
fun register(
@HeaderParam(value = "X-Auth-Token") token: String?,
@HeaderParam(value = "X-Account-Meta-Temp-URL-Key") tempUrlKey: String?,
@HeaderParam(value = "X-Account-Meta-Temp-URL-Key-2") tempUrlKey2: String?
): ResponseWe use two different tempUrlKeys so that we can achieve a simple load-balancing behavior.
private fun getTempURLSecretKey(): String {
if (ZonedDateTime.now(ZoneOffset.UTC).second % 2 == 0) {
return ossConfiguration.tempUrlSecretKey()
}
return ossConfiguration.tempUrlSecretKey2()
}Now we are registering with this token:
fun register() {
try {
ossClient.register(ossAuthService.getToken(),
ossConfiguration.tempUrlSecretKey(),
ossConfiguration.tempUrlSecretKey2())
} catch (e: Exception) {
LOGGER.log(
Logger.Level.ERROR,
e.message,
e
)
}
}Then we can create our temp url:
fun createTempUrl(path: String?): String? {
val now = ZonedDateTime.now(ZoneOffset.UTC)
val expires = now
.plus(Duration.ofMinutes(ossConfiguration.ttlInMinutes()))
val expiresEpoch = expires.toEpochSecond()
val modifiedPath = "/swift/v1$path"
val hmacBody = join("\n", HttpMethod.GET, expiresEpoch.toString(), modifiedPath)
val hmac = HmacUtils(HmacAlgorithms.HMAC_SHA_1, getTempURLSecretKey())
val signature: String = hmac.hmacHex(hmacBody)
return format(
"%s%s?temp_url_sig=%s&temp_url_expires=%s",
ossConfiguration.ossPath(),
modifiedPath,
signature,
expiresEpoch
)
}Now influencers can view their invoices for a temporary url that they create for 5 minutes, and can re-request their past invoices when they want to see them again.
The result is like this:
Example Go code:
type ossAuthService struct {
cron *gocron.Cron
token string
httpClient http_adapter.Client
ossConfig config.OssConfig
}
type CacheService interface {
GetToken() string
}
func NewOssAuthService(cron *gocron.Cron, token string, ossConfig config.OssConfig, httpClient http_adapter.Client) CacheService {
return &ossAuthService{cron: cron, token: token, httpClient: httpClient, ossConfig: ossConfig}
}
func (service ossAuthService) GetToken() string {
token := service.token
if token == "" {
service.refreshToken()
token = service.token
}
return token
}
func (service ossAuthService) addScheduled() {
service.refreshToken()
_, _ = service.cron.AddFunc("@every 24h", func() {
service.refreshToken()
})
}
func (service ossAuthService) refreshToken() {
request := buildTokenRequest(service.ossConfig.Id, service.ossConfig.Secret)
token, err := service.fetchToken(nil, request)
if err != nil {
logger.Logger().Error("token refresh failed, err: " + err.Error())
return
}
service.token = token
}
func (service ossAuthService) fetchToken(c *gin.Context, request string) (string, error) {
response, err := service.httpClient.Post(c, service.ossConfig.AuthUrl,
"/v3/auth/tokens", request)
if err != nil {
return "", err
}
tokenHeaderValue := response.Header.Peek("x-subject-token")
if tokenHeaderValue == nil {
return "", err
}
return string(tokenHeaderValue), nil
}
func buildTokenRequest(id string, secret string) string {
return "{\n" +
" \"auth\":{\n" +
" \"identity\":{\n" +
" \"methods\":[\n" +
" \"application_credential\"\n" +
" ],\n" +
" \"application_credential\":{\n" +
" \"id\":\"" + id + "\",\n" +
" \"secret\":\"" + secret + "\"\n" +
" }\n" +
" }\n" +
" }\n" +
"}"
}func (service *ossService) Register(c *gin.Context) error {
token := service.ossAuthCacheService.GetToken()
headers := map[string]string{"X-Auth-Token": // token
"X-Account-Meta-Temp-URL-Key": // tempUrlSecretKey
"X-Account-Meta-Temp-URL-Key-2": // tempUrlSecretKey2
}
return service.httpClient.Post(c, service.ossConfig.Url, nil, headers)
}func (service invoiceService) CreateTempURL(bucket string, path string) UrlResponse {
duration := time.Duration(300) * time.Second
expiry := time.Now().Add(duration).Unix()
secretKey := []byte(getTempURLSecretKey())
objectPath := fmt.Sprintf("/swift/v1/%s/%s", bucket, path)
body := fmt.Sprintf("%s\n%d\n%s", "GET", expiry, objectPath)
hash := hmac.New(sha1.New, secretKey)
hash.Write([]byte(body))
hexsum := fmt.Sprintf("%x", hash.Sum(nil))
tempUrl := fmt.Sprintf("%s%s?temp_url_sig=%s&temp_url_expires=%d",
"https://oss.example.com",
objectPath, hexsum, expiry)
return UrlResponse{Url: tempUrl}
}
func getTempURLSecretKey() string {
if time.Now().Nanosecond() % 2 == 0 {
// return tempUrlSecretKey
}
// return tempUrlSecretKey2
}The reason me and Emre Tanrıverdi wrote this story is to lend those, who want to do a similar thing in their projects, a helping hand.
We hope it was helpful. :)
Thank you for reading! ❤️
Thanks to all our colleagues in the Influencer Communications & Favorites Team. 🤟
