SOLR-18248 : Migration of Tasks V2 APIs to JAX-RS Constructs#4452
SOLR-18248 : Migration of Tasks V2 APIs to JAX-RS Constructs#4452jaykay12 wants to merge 35 commits into
Conversation
|
|
||
| import static org.apache.solr.client.api.util.Constants.INDEX_PATH_PREFIX; | ||
|
|
||
| @Path(INDEX_PATH_PREFIX + "/tasks/listjalaz") |
There was a problem hiding this comment.
;-) Look slike you have some debugging?
There was a problem hiding this comment.
yeah, started with firstly wiring this api to give any random response, & now using this to compare my V2 response with the original V2 response, 😄 thus, kept a different api endpoint for now to do response comparision.
will surely update this during self-review before raising for formal review 💯
epugh
left a comment
There was a problem hiding this comment.
Thanks for getting started on this, I went ahead and put some comments in that may be too early to be added ;-)
| @GET | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Lists all the currently running tasks or status of any taskUUID being passed as queryParam", |
There was a problem hiding this comment.
So, I think we would revamp this api... if you look at the desired api, ListActiveTasks would just list all tasks. If you wanted a specfific task, then it would be a idfferent end point with a /taskUUID.
There was a problem hiding this comment.
yes, have updated this Api class for now, handling the taskUUID as a path param now, it was queryParam in the V1 as well as homegrown v2.
|
|
||
| public class ListActiveTaskResponse extends SolrJerseyResponse { | ||
| @JsonProperty | ||
| public Map<String, String> taskList; |
There was a problem hiding this comment.
We are trying to move to more strongly typed responses so taht we don't have Map everywhere. Go ahead and specify out the return properties!
There was a problem hiding this comment.
Yes, have done so.
|
|
||
| } | ||
|
|
||
| private void handleStandAloneMode(ListActiveTaskResponse response, String taskUUID) { |
There was a problem hiding this comment.
One thing we are trying to do is move the V1 API logic into the V2 code base, and have the V1 call the V2 version, this way, as these apis evolve, we know that v1 and v2 stay in sync till we get rid of V1. I believe that ideally we would want ActiveTasksListComponent to delegate to the same code as this V2 api. Now, maybe it's so simple that both can just call core getCancellableQueryTracker().getActiveQueriesGenerated. In other V2 apis, you can see the code live in the V2 and then V1 calls it...
There was a problem hiding this comment.
understood this part, have made suitable changes.
| log.debug("solr cloud"); | ||
| } | ||
| handleSolrCloudMode(response, taskUUID); | ||
| } else { |
There was a problem hiding this comment.
not quite sure I see the differences in the two paths yet?
There was a problem hiding this comment.
yea, i am also checking this up.
I have tested this endpoint by running on cloud example which we have in examples, it worked fine. It should work fine for standalone as well, will remove the current conditional branching done once that's verified.
thank you so much @epugh for the prompt reviews & initial guidance, much appreciated. I will keep addressing your comments as we go, I kept this PR as draft for now as per our community guidelines, but yea thanks again for reviewing this at such initial state, so that i can course correct myself promptly. ✨ |
epugh
left a comment
There was a problem hiding this comment.
Good progress. There are definitly some changes to make this more restful, and to reuse our SolrJ methods in the tests. This appears to currently focus on listing of tasks and status of a single task, I don't see cancellation supported. That's just fine, we can do that one in a seperate PR!
| import org.apache.solr.client.api.util.StoreApiParameters; | ||
|
|
||
| @Path(INDEX_PATH_PREFIX + "/tasks/list") | ||
| public interface ListActiveTasksApi { |
There was a problem hiding this comment.
I am looking at the org.apache.solr.client.api.endpoint.ConfigSetsApi and liking it's pattern a bit better. I'll try and provide some detailed feedback! I think there is an opportunity to have line 30 be /tasks
There was a problem hiding this comment.
took reference of the ConfigSetsApi, & have changed the contracts accordingly.
| @Path(INDEX_PATH_PREFIX + "/tasks/list") | ||
| public interface ListActiveTasksApi { | ||
|
|
||
| // Handles: .../tasks/list (Lists all) |
There was a problem hiding this comment.
i don't think we need this comment.
There was a problem hiding this comment.
have removed this.
| import org.apache.solr.client.api.model.TaskStatusResponse; | ||
| import org.apache.solr.client.api.util.StoreApiParameters; | ||
|
|
||
| @Path(INDEX_PATH_PREFIX + "/tasks/list") |
There was a problem hiding this comment.
if the path here is /tasks then we don't need the /list, as the pattern is GET /tasks will list all the tasks. This is more restful.
There was a problem hiding this comment.
understood, have addressed.
| import org.apache.solr.client.api.util.StoreApiParameters; | ||
|
|
||
| @Path(INDEX_PATH_PREFIX + "/tasks/list") | ||
| public interface ListActiveTasksApi { |
There was a problem hiding this comment.
yes, makes sense. have addressed this.
| tags = {"tasks"}) | ||
| ListActiveTaskResponse listAllActiveTasks() throws Exception; | ||
|
|
||
| // Handles: .../tasks/list/slow-task-id (Lists specific) |
There was a problem hiding this comment.
i don't think we need this comment, and I think this should be GET /tasks/taskId
There was a problem hiding this comment.
done., comment removed. GET endpoint updated.
There was a problem hiding this comment.
I am expecting a DELETE /tasks/{taskId} for cancelling a running task. Or... we need a PATCH that does the cancel PATCH /tasks/{taskId}/cancel
There was a problem hiding this comment.
yes, in this PR scope, #4452 (comment) took only 2 out of 3 API related to Tasks.
Cancel Running task API migration will be a follow-up, independent PR of itself.
| public ActiveTaskDetails() {} | ||
|
|
||
| public ActiveTaskDetails(String taskUUID, String taskQuery) { | ||
| this.taskUUID = taskUUID; |
There was a problem hiding this comment.
you know, maybe we just keep taskUUID for now... just to reduce churn. Even though I don't like it as a name.
There was a problem hiding this comment.
have addressed this.
| import com.fasterxml.jackson.annotation.JsonProperty; | ||
|
|
||
| public class TaskStatusResponse extends SolrJerseyResponse { | ||
| @JsonProperty public String taskStatus; |
There was a problem hiding this comment.
maybe an oppotunity to use an enum for the various taskStatus????
There was a problem hiding this comment.
also, I am getting the feeling that taskStatus isnt' actually a good format as currently defined. I looked int he docs: { "responseHeader":{ "status":0, "QTime":16}, "taskStatus":"id:5, status: active"}
and at a minimum, it should just be:
{
"responseHeader":{
"status":0,
"QTime":16},
"taskStatus":"active"}
because we already know the id. Now, if we are listing, we should have something like:
{
"responseHeader":{
"status":0,
"QTime":16},
"tasks":[
"taskId":"5", "status":"active"}
]
}
There was a problem hiding this comment.
maybe an oppotunity to use an enum for the various taskStatus????
yes, thanks. have made taskStatus as enum.
Now, if we are listing, we should have something like:
we are not returning list, in case of getTaskStatus API, its taskId & the status in String format in v1 API, in V2 we will be returning just the taskStatus enum.
| params.set("taskUUID", "5"); | ||
|
|
||
| var request = | ||
| new GenericSolrRequest( |
There was a problem hiding this comment.
When I see GSR, generally that is an opportunity for us to use our SolrJ generated API instead. This gets us testing of the SolrJ response code for free!
There was a problem hiding this comment.
i added these 2 extra tests over here, since i felt earlier added assertions were not adequate enough. They were not covering GetStatusTaskAPI fully.
This test validates for the v1 api flow though, is what i feel.
|
|
||
| @Test | ||
| public void testCheckSpecificQueryStatus_Inactive() throws Exception { | ||
| for (int i = 0; i < 10; i++) { |
There was a problem hiding this comment.
why do we need to execute 10 of these? Seems like a magic number!
There was a problem hiding this comment.
not a magic number, i am just executing 10 queries during test & since each query gets fired with their index as taskID, so i am simply asserting for any taskId=5 which should be active since it is <10 & asserting for taskId=15, which should be inactive since its outside the range.
this number can be anything, should i keep this lower, like 2 or 3?
epugh
left a comment
There was a problem hiding this comment.
Some great progress, I think we are getting closer.
One question, have you been able to fire up Solr and test these apis manually? I have a slight fear that the overall Task Management API might not be working at all ;-(. It would be nice to confirm in a running Solr that yes, this is all working.
| @Path(INDEX_PATH_PREFIX + "/tasks/list") | ||
| public interface ListActiveTasksApi { | ||
| @Path(INDEX_PATH_PREFIX) | ||
| public interface TasksApi { |
| @Operation( | ||
| summary = "Status of a specific taskID passed as pathParam.", | ||
| tags = {"tasks"}) | ||
| TaskStatusResponse getTaskStatus(@PathParam("taskID") String taskID) throws Exception; |
There was a problem hiding this comment.
We should remember to rationalize all our various "id" related attributes to have a similar pattern... across all the v2 apis... ;-)
| @GET | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Lists all the currently running active tasks.", |
There was a problem hiding this comment.
I wonder if this should be Lists all the currently active tasks.. Running/not running, that really isn't an aspect. The task is either active or not, at least, that is how I read the TaskStatus enum! Nit picky, but now is the time.
There was a problem hiding this comment.
have updated the summary to now: Lists all the active tasks.
Here via this API call, we invoke: getActiveQueriesGenerated(), thus keeping active fits here.
| @GET | ||
| @StoreApiParameters | ||
| @Operation( | ||
| summary = "Status of a specific taskID passed as pathParam.", |
There was a problem hiding this comment.
I don't think we need the passed as pathParam, that is obvious from thow the docs are generated.. In fact, I would do Status of a specific task. and leave it at that.
There was a problem hiding this comment.
understood, have corrected this now.
|
|
||
| response.taskStatus = | ||
| (isTaskActive) | ||
| ? TaskStatusResponse.TaskStatus.ACTIVE |
There was a problem hiding this comment.
I think I would just import TaskStatusResponse to have a less nested strcutrure.
There was a problem hiding this comment.
actually, we have that, so can we just refer to TaskSTatus?
There was a problem hiding this comment.
thanks, this looks much better now, lesser visual noise.
| import org.apache.solr.request.SolrQueryRequest; | ||
|
|
||
| public class ListActiveTasks extends JerseyResource implements ListActiveTasksApi { | ||
| public class ListActiveTasks extends JerseyResource implements TasksApi.List { |
| * Handles request for listing all active cancellable tasks All active tasks logic lives in the v2 | ||
| * {@link ListActiveTasks}; this handler is a thin v1 bridge that extracts request parameters and | ||
| * delegates. | ||
| * Handles request for listing the active cancellable tasks and Status check of any particular task, |
There was a problem hiding this comment.
are there active tasks that are NOT cancellable?
There was a problem hiding this comment.
also, maybe a typo of "Status" check? Lastly, this whole sentence maybe needs a period? Just appears a bit awkared.
In fact, i think we don't need the text about "actual logic", i don't think we do that in other apis? If we do, then great...
There was a problem hiding this comment.
are there active tasks that are NOT cancellable?
there can be. since this is the kind of function that we invoke, having cancellable in the javadoc looks fine to me.
getCore().getCancellableQueryTracker().getActiveQueriesGenerated()
| @Override | ||
| public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws Exception { | ||
| String taskStatusCheckUUID = req.getParams().get(TASK_CHECK_UUID, null); | ||
| String taskStatusCheckID = req.getParams().get(TASK_CHECK_UUID, null); |
There was a problem hiding this comment.
so, we can't change any signatures of v1, so I don't know if TASK_CHECK_UUID changed? I think it's fine to leave "taskStatusCheckUUID", as we hope to get rid of it? Maybe just a comment? I dunno.
Or, we change taskStatusCheckID to just taskID internally if we are going to rename it?
There was a problem hiding this comment.
have reverted this back to taskStatusCheckUUID
jaykay12
left a comment
There was a problem hiding this comment.
code comments addressed ✅
local verification of v1 & v2 APIs - ToBeAttached ❗
| (isTaskActive) | ||
| ? TaskStatusResponse.TaskStatus.ACTIVE | ||
| : TaskStatusResponse.TaskStatus.INACTIVE; | ||
| response.taskStatus = (isTaskActive) ? TaskStatus.ACTIVE : TaskStatus.INACTIVE; |
Here is an example of sort of the testing I am looking for. In the reference PR I found a couple of commands that exercised the code externally, just to get more confidence. If you do the equivalent with some curl commands, that would give me more confidence. Plus, I'll run them and learn some stuff! #4320 (comment) |
|
@jaykay12 oh, we need to update the Reference Guide for any changes we've made in the V2 outputs as well ;-). If you hate doing that, I can do that ;-). |
Turns out I started this pattern, but it's not actually a pattern we want to support! So backing it out of the two places that have it.
https://issues.apache.org/jira/browse/SOLR-18248
Description
Migrating 2 out of the 3 V2 APIs of Tasks, from existing homegrown annotations to JAX-RS Standard constructs & annotations
Solution
Have exposed 1 new interface: ListActiveTasksApi which serves the
/lists/taskas well as/list/task/<taskId>endpoints.ListActiveTasks is the main class which takes on the responsibility for v2 path as well as also for the v1 api path via ActiveTasksListHandler
2 new DTO for List Task API
1 new DTO for Task Status API
Removed the now deprecated files: ListActiveTasksAPI & ActiveTasksListComponent from the codebase.
No Coding Agent used.
Tests
Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem.
Checklist
Please review the following and check all that apply:
mainbranch../gradlew check.