Looking for urgent help with API

Yeah I noticed that. The default responses all pretty much follow the same pattern. I'll post some samples tomorrow of how I'm doing things with posts, puts, gets, deletes etc and hopefully you can provide some feed back. Not now though. Time for a movie I reckon. Thanks @_kabal_
 
Chances is good that you will need to micro-service this. I would have from the start. Then you need Dapper and don`t need the repo pattern or EF. My opinion is that a repo pattern is less needed in a micro-service. I would say the possibility to go micro-service needs to be kept in mind today, always.

Edit: Maybe these days a micro-service is the only option.
 
DA-LION-619 said:
If you’re only using SQL Server (let’s be honest), which isn’t cheap then use it.
I’d let the SQL Server handle auditing if needed as it can clean it up and also provides flexibility in picking languages, frameworks etc.
Click to expand...
So tell me junior, what is cheap compared to SQL Server? MySQL?????:ROFL::ROFL::ROFL::sleep:
 
I have a question for you guys please. Generating a C# rest client with NSwag using OpenApi specs on my controllers.

Now one of my API controllers has an IFormFile parameter for uploading a file.

When I generate the client in NSwag and use this library in an MCV project, the IFormFile is replaced with something called FileParameter in the generated client library.

Ive tried converting the IFormfile on the MVC side to this type to post it to the API but gave up as I concluded that this is not the right way.

I went back to my API side and changed IFormFile parameter to a byte array and concluded that this is also not the right way as many tutorials on API's use IFormFile for posting to an API.

So I'm officially stumped. Some guidance would be greatly appreciated.
 
what does your openAPI requestBody look like

I've only used this from Typescript side, but it works fine

Open API
JSON: 
   "/api/files": {
      "post": {
        "tags": [
          "Files"
        ],
        "operationId": "Files_UploadFile",
        "requestBody": {
          "content": {
            "multipart/form-data": {
              "schema": {
                "type": "object",
                "properties": {
                  "File": {
                    "type": "string",
                    "format": "binary",
                    "nullable": true
                  }
                }
              }
            }
          }
        },
        "responses": {
          "204": {
            "description": ""
          }
        }
      }
    },

Typescript
JavaScript: 
uploadFile: async(file: FileParameter | null | undefined, abortSignal?: AbortSignal | undefined) : Promise<void> => {
    let url_ = "/api/files";
    url_ = url_.replace(/[?&]$/, "");
    const content_ = new FormData();
    if (file !== null && file !== undefined)
        content_.append("File", file.data, file.fileName ? file.fileName : "File");
    let config_: AxiosRequestConfig = {
        data: content_,
        method: "POST",
        url: url_,
        headers: {
        },
        signal: abortSignal
    };

    const { data } = await axiosToUse.request(config_).catch(error => Promise.reject(handleError(error)));
    return data as void;
},

export interface FileParameter {
    data: any;
    fileName: string;
}

Note - FileParameter is just a made up type/interface so that the API Client can be typed.
 
Thanks kabal. Will pop back in a bit after visitors leave and post my request body.
 
I assume you want to do this.

  • Upload File in your MVC app
  • Then Use API Client To call another API, sending the IFormFile data to the API
This is easy :)

C#: 
// I am assuming that FileParameter's Data property is of type "byte[]"

await using var fileStream = formFile.OpenReadStream();
using var memoryStream = new MemoryStream();

await fileStream.CopyToAsync(memoryStream, cancellationToken);

var fileContent = memoryStream.ToArray();

var fileParameter = new FileParameter
{
    Data = fileContent,
    Filename = formFile.Filename
}
 
_kabal_ said:
I assume you want to do this.

  • Upload File in your MVC app
  • Then Use API Client To call another API, sending the IFormFile data to the API
This is easy :)

C#: 
// I am assuming that FileParameter's Data property is of type "byte[]"

await using var fileStream = formFile.OpenReadStream();
using var memoryStream = new MemoryStream();

await fileStream.CopyToAsync(memoryStream, cancellationToken);

var fileContent = memoryStream.ToArray();

var fileParameter = new FileParameter
{
    Data = fileContent,
    Filename = formFile.Filename
}
Click to expand...
Oh dear lord files and memory streams......having flashbacks....like back in Nam.

Not moaning about just been in the trenches with it. Keep in mind restrictions ( web request length etc ) and the size and impact to your api with uploading files. Also, if I remember correctly there is a hard/soft limit for basic file uploading to be around ~40ish Mb. You need/should implement segmented uploading or ideally uploading to a proper blob repo ( Azure Storage / Amazone S3 etc ) and linking so as not to cause hiccups/hangs on your apis, nor rack up extra processing unnecessarily.
 
Kosmik said:
Oh dear lord files and memory streams......having flashbacks....like back in Nam.

Not moaning about just been in the trenches with it. Keep in mind restrictions ( web request length etc ) and the size and impact to your api with uploading files. Also, if I remember correctly there is a hard/soft limit for basic file uploading to be around ~40ish Mb. You need/should implement segmented uploading or ideally uploading to a proper blob repo ( Azure Storage / Amazone S3 etc ) and linking so as not to cause hiccups/hangs on your apis, nor rack up extra processing unnecessarily.
Click to expand...

100% both AWS and azure make this really easy with signed URLs.

1670949063812.png

Quite easy to implement the above in .Net too @Solarion

Something to think about once you have a working solution. Depends where/how you host your application and what storage you have
 
_kabal_ said:
100% both AWS and azure make this really easy with signed URLs.

View attachment 1441041

Quite easy to implement the above in .Net too @Solarion

Something to think about once you have a working solution. Depends where/how you host your application and what storage you have
Click to expand...
Yeah , we do large video uploads like that. Setup api call, segmented upload to azure blob storage, post api call with storage result and background services then run for processing/transcoding etc. Low api overhead, highly scalable and resilient.
 
_kabal_ said:
what does your openAPI requestBody look like
Click to expand...

Not quite like yours, something is off. See from properties. My content type is a string whereas yours is a file and there are a whole bunch of other properties in the openapi requestbody that I don't think should be there.

Also now on the client side when I try to make this call, I'm getting a whole bunch of other parameters I have no idea what to do with and the option to use a FileParameter has vanished :/

uploadasync.jpg

Code: 
 "/api/databases/upload-clean-database": {
      "post": {
        "tags": [
          "Database"
        ],
        "summary": "Uploads a clean database",
        "requestBody": {
          "content": {
            "multipart/form-data": {
              "schema": {
                "type": "object",
                "properties": {
                  "ContentType": {
                    "type": "string"
                  },
                  "ContentDisposition": {
                    "type": "string"
                  },
                  "Headers": {
                    "type": "object",
                    "additionalProperties": {
                      "type": "array",
                      "items": {
                        "type": "string"
                      }
                    }
                  },
                  "Length": {
                    "type": "integer",
                    "format": "int64"
                  },
                  "Name": {
                    "type": "string"
                  },
                  "FileName": {
                    "type": "string"
                  }
                }
              },
              "encoding": {
                "ContentType": {
                  "style": "form"
                },
                "ContentDisposition": {
                  "style": "form"
                },
                "Headers": {
                  "style": "form"
                },
                "Length": {
                  "style": "form"
                },
                "Name": {
                  "style": "form"
                },
                "FileName": {
                  "style": "form"
                }
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Uploaded database successfully",
            "content": {
              "application/json": {
                "schema": {
                  "type": "boolean"
                }
              }
            }
          },
          "400": {
            "description": "Failed to upload database",
            "content": {
              "application/json": {
                "schema": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
    },

Controller method

Controller.jpg
 
That is strange, but it shouldn’t really matter. Could just be a version difference.
What does the FileParameter look like?

What happens if you load this openapi spec into postman and call it from there?
 
_kabal_ said:
That is strange, but it shouldn’t really matter. Could just be a version difference.
What does the FileParameter look like?

What happens if you load this openapi spec into postman and call it from there?
Click to expand...

Gonna try that after loadshedding in a bit thanks kabal
 
@_kabal_ as a matter of interest, why does one need signed url? For Azure i use FileshareClient or BlobClient in the webapi backend and then write to that as a stream. Is it so that a client can write directly to storage as opposed to via webapi?
 
Spacerat said:
@_kabal_ as a matter of interest, why does one need signed url? For Azure i use FileshareClient or BlobClient in the webapi backend and then write to that as a stream. Is it so that a client can write directly to storage as opposed to via webapi?
Click to expand...
That’s exactly it.
It’s not a huge issue if you are on bare metal, and your users aren’t uploading large files - you basically have unlimited cpu + memory.

But imagine running with a 0.25 vCPUs and 512/1024MB memory (fargate/azure equivalent). You’ll potentially quickly
run into issues trying to handle multiple users/uploads.
Getting a signed URL is fast/cheap.
And S3/Azure equivalent bandwidth/scaling can be seen as practically “infinite”

You can also potentially avoid ingress -> egress -> ingress costs, which could realistically happen if users are uploading GB’s of files


It also essentially does the same for serving files. When a user wants to download a file, check if they are allowed to download that file, get a signed url, and return a 302 with the signed url.

The signed urls are also only valid for x amount of time
 
Last edited:
_kabal_ said:
That is strange, but it shouldn’t really matter. Could just be a version difference.
What does the FileParameter look like?

What happens if you load this openapi spec into postman and call it from there?
Click to expand...

I get this when importing into Postman. So the nswag client is showing the right information, I'm just not sure what to do with this lol! I would have expected maybe a file open dialog box. I'm going to try by removing the IFormFile parameter from the api endpoint and just post it like in mvc. Not sure if that would help.

Screenshot_2.jpg


It's visible on the binary tab.

Screenshot_3.jpg

I basically have no idea how to even call this from a client side. The only thing I can think of is that I'm configuring the CSharp client incorrectly in Nswag studio.
 
Last edited:
Ok to get this to work I had to wrap IFormFile in a class. Either that or else deal with like 6 parameters on the Client side which I think represent or are supposed to represent an IFormFile object. Odd.

Client

Screenshot_5.jpg

API Side

Screenshot_6.jpg

Screenshot_7.jpg
 
_kabal_ said:
That’s exactly it.
It’s not a huge issue if you are on bare metal, and your users aren’t uploading large files - you basically have unlimited cpu + memory.

But imagine running with a 0.25 vCPUs and 512/1024MB memory (fargate/azure equivalent). You’ll potentially quickly
run into issues trying to handle multiple users/uploads.
Getting a signed URL is fast/cheap.
And S3/Azure equivalent bandwidth/scaling can be seen as practically “infinite”

You can also potentially avoid ingress -> egress -> ingress costs, which could realistically happen if users are uploading GB’s of files


It also essentially does the same for serving files. When a user wants to download a file, check if they are allowed to download that file, get a signed url, and return a 302 with the signed url.

The signed urls are also only valid for x amount of time
Click to expand...
Noted on the effective use of resources. You don't want to bog down your resource by just moving bytes into storage. My largest file being uploaded is 60MB.. so prob not a huge issue.
Not 100% understanding your "ingress -> egress -> ingress". If the app and the webapi is hosted in azure then the app=>api would likely not count as egress? Or is my assumption wrong?
 
@Solarion , not sure what you do with the files after upload, but make sure that you separate the upload from any processing. Processing can take long (depending on what you do) and can easily make the upload request time out. I upload the file to Azure file share or blob storage and then drop a message into an Azure queue. This message gets picked up by a Azure queue-triggered function and then does the processing in the background, free from time limits.

This is what my upload looks like on back-end
1671009428555.png
The request
1671009466719.png

And the MediaTR handler

1671009615770.png
Methods:
1671009697287.png

AzureFileShareFileStore:
1671009782630.png

Hmm looking at it now, I might need to dispose/using the stream I get from formFile.OpenReadStream

I have some client code to do the upload. Just need to find it
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X